UEL¶

The Unscrambl Qbo Decisions Expression Language (UEL)¶

Introduction¶

The Unscrambl Qbo Decisions Expression Language (UEL) is a simple expression language used to specify filter conditions and basic arithmetic manipulations. It can be used as part of configuring triggers. In particular, a trigger’s WHEN condition can have a free-form expression specified as part of a predicate’s right hand-side. Similarly, a trigger’s THEN action can have an expression specified as an assignment for an output attribute. These expressions are specified in UEL.

Types¶

An expression in UEL can have one of the following primitive types: String (a string), Bool (a Boolean), Int16 (a 16-bit integer), Int32 (a 32-bit integer), Int64 (a 64-bit integer), or Double (a double precision floating point number). It can also have a list type, where the element type of the list is one of the primitive types: List(String), List(Bool), List(Int16), List(Int32), List(Int64), List(Double).

Literals¶

Bool, Double, Int32, Int64, and String literals can be present in an expression.

A Bool literal can be either true or false.
A Double literal is a decimal number that either contains the decimal separator (e.g., 3.5, .5, 3.) or is given in the scientific notation (e.g., 1e-4, 1.5e3). A Double literal must be between (2 - 2 ^ -52) * 2 ^ 1023 (~ 1.79 * 10 ^ 308) and -(2 - 2 ^ -52) * 2 ^ 1023 (~ -1.79 * 10 ^ 308). Moreover, the magnitude of a literal cannot be less than 2 ^ -1074 (~ 4.94 * 10 ^ -324). Furthermore,
- a NaN (not a number) value can be specified through the Double("nan") expression,
- an Inf (positive infinity) value can be specified through the Double("inf") expression,
- and a -Inf (negative infinity) value can be specified through the Double("-Inf") expression.
Integer literals without a suffix are of the Int32 type (e.g., 14). An Int32 literal must be between 2 ^ 31 - 1 (= 2147483647) and -2 ^ 31 (= -2147483648).
The L suffix is used to create Int64 literals (e.g., 14L). An Int64 literal must be between 2 ^ 63 - 1 (= 9223372036854775807L) and -2 ^ 63 (= -9223372036854775808L).
A String literal appears within double quotes, as in "I'm a string literal". The escape character \ can be used to represent new line (\n), tab (\t), double quote (\") characters, as well as the escape character itself (\\).

Arithmetic Operations¶

An expression in UEL can contain the basic arithmetic operations: addition (+), subtraction (-), multiplication (*), division (/), and modulo (%) with the usual semantics. These are binary operations that expect sub-expressions on each side. An expression in UEL can also contain the unary minus (-) operation that expects a sub-expression on the right. Finally, parentheses (()) are used for adjusting precedence, as usual.

Addition operation corresponds to concatenation for the String type and is the only available operation on this type. Bool type does not support arithmetic operations. Division applies integer division on integer types and floating point division on Double types.

The modulo operation yields the remainder from the division of the first operand by the second. It always yields a result with the same sign as its second operand. The absolute value of the result is strictly smaller than the absolute value of the second operand.

Examples:

A simple arithmetic expression: (3 + 4 * 5.0) / 2
A simple expression involving a String literal: "area code\tcountry"
A unary minus expression: -(3 + 5.0)

Comparison Operations¶

An expression in UEL can contain the basic comparison operations: greater than (>), greater than or equal (>=), less than (<), less than or equal (<=), equals (==), and not equals (!=) with the usual semantics. These are binary operations. With the exception of equals and not equals, they expect sub-expressions of numerical types or strings on each side. For strings, the comparison is based on lexicographic order. For equals and not equals, the left and the right sub-expressions must be of compatible types with respect to UEL coercion rules. The result of the comparison is always of type Bool.

Examples:

An expression using comparisons: 3 > 5
An expression using String comparisons: "abc" < "def"
An expression using inequality comparison: "abc" != "def"
An expression comparing a Double literal with a NaN (not a number) value: 10.5 != Double("nan") (Note that in this case, the math.isNaN built-in function can also be used.)
An expression comparing a Double literal with an Inf (positive infinity) value: 10.5 != Double("inf") (Note that in this case, the math.isPositiveInfinity built-in function can also be used.)
An expression comparing a Double literal with a -Inf (negative infinity) value: 10.5 != Double("-inf") (Note that in this case, the math.isNegativeInfinity built-in function can also be used.)

Logical Operations¶

An expression in UEL can contain the basic logical operations: logical and (&&) and logical or (||) with the usual semantics. These are binary operations that expect sub-expressions of type Bool on each side. Shortcutting is used to evaluate the logical operations. For logical and, if the sub-expression on the left evaluates to false, then the sub-expression on the right is not evaluated and the result is false. For logical or, if the sub-expression on the left evaluates to true, then the sub-expression on the right is not evaluated and the result is true. An expression in UEL can also contain the not (!) operator, which is a unary operator that expects a sub-expression of type Bool on the right.

Examples:

A simple logical expression: 3 > 5 || 2 < 4
A logical expression that uses not: ! (3 > 5)

Ternary Operation¶

An expression in UEL can make use of the the ternary operation: condition ? choice1 : choice2. The condition of the ternary operation is expected to be of type Bool and the two choices are expected to be sub-expressions with compatible types. The ternary operation is lazily evaluated. If the condition evaluates to true, then the result is the first choice, without the sub- expression for the second choice being evaluated. If the condition evaluates to false, then the result is the second choice, without the sub-expression for the first choice being evaluated.

Examples:

A simple ternary operation: 3 < 10 ? "smallerThan10" : "notSmallerThan10"

List Operations¶

A list is constructed by specifying a sequence of comma (,) separated values of the element type, surrounded by brackets ([]) . For instance, an example literal for List(Double) is [3.5, 6.7, 8.3] and an example for List(String) is ["Unscrambl", "Qbo Decisions"]. An empty list requires casting to define its type. As an example, an empty List(String) can be specified as List(String)([]).

An expression in UEL can contain a few basic list operations: containment (in), non-containment (not in), indexing ([i]), slicing ([i:j]), concatenation (+), and size inquiring (size()).

Containment yields a Boolean answer, identifying whether an element is contained within a list. For instance, 3 in [2, 5, 3] yields true, whereas 8 in [2, 5, 3] yields false. Non-containment is the negated version of the containment. For instance, 3 not in [2, 5, 3] yields false, whereas 8 not in [2, 5, 3] yields true.

Indexing yields the element at the specified index within the list. 0-based indexing is used and indices are of type Int32. For instance, [2, 5, 3][1] yields 5, and [2, 5, 3][-3] yields 2. The index should be between -size and size-1, inclusive. An index that is out of these bounds will result in an evaluation error at runtime.

Slicing yields a sub-list. The start index is inclusive, whereas the end index is exclusive. If the range is out of bounds, then an empty list is returned. For instance, [2, 5, 3, 7][1:2] yields [5], [2, 5, 3, 7][1:3] yields [5, 3], [2, 5, 3, 7][1:1] yields [], [2, 5, 3, 7][3:5] yields [7], [2, 3, 4][-2:-1] yields [3], [2, 3, 4][-5:-1] yields [2, 3], [2, 3, 4][1, 6] yields [3, 4], and [2, 5, 3, 7][4:6] yields [].

Concatenation results in a list that contains the elements from the first list followed by the elements from the second list. For instance, [1, 2] + [3] yields [1, 2, 3].

The size of a list can be retrieved via the builtin function list.size().

Precedence and Associativity of Operations¶

Operations	Associativity
`()`
`[]`
unary `-`, `!`
`*`, `/`, `%`	left
`+`, `-`	left
`>`, `>=`, `<`, `<=`	left
`==`, `!=`	left
`in`
`&&`	left
`\|\|`	left
`?:`

Attributes¶

Expressions in UEL can also contain attributes. Attributes are identifiers that correspond to the attributes available in a tuple or in the master profile. Each attribute has a type and can appear in anywhere a sub-expression of that type is expected.

If an attribute comes from the master profile, it can be referenced by prefixing it with profile.. Otherwise, only the attribute name is used.

Examples:

A string formed by concatenating a String literal and an attribute named code from the current tuple: "AREA_" + code
A string formed by concatenating a String literal and a profile attribute named name: "My name is " + profile.name
An arithmetic expression involving an attribute and Int32 literals: numSeconds / (24 * 60 * 60)
An arithmetic expression involving a profile attribute and Int32 literals: profile.ageInSeconds / (24 * 60 * 60)
A floating-point arithmetic expression, where the floating point literal is of type Double: cost / 1000.0
Another expression involving a profile attribute, where the floating point literal is of type Double: profile.revenue / 1000.0
A Boolean expression checking if a String literal is found in an attribute named places of type List(String): "airport" in places
A Boolean expression checking if a String literal is found in a profile attribute named favoritePlaces of type List(String): "airport" in profile.favoritePlaces

Conversions¶

UEL supports explicit conversions via casts using a function call syntax. The name of the function is the name of the type we want to cast to.

Examples:

Casting an Int32 to a String: String(14) yields "14"
Casting a String to a Double: Double("4.5") yields 4.5
Casting a String to a Double: Double("nan") yields NaN (not a number)

UEL supports implicit conversions (aka coercions) as well. When two integers of different types are involved in an operation, the one that has the smaller number of bits is coerced into the wider type. When an integer is involved in an operation with a Double it is coerced into a Double. Also note that, in such operations, Int64 values that cannot be represented exactly will be rounded to the closest Double value.

Examples:

Coercion with integers of different bit-lengths: count / 2 has type Int64, assuming count is of type Int64 (this has the same semantics as the expression count / Int64(2))
Coercion involving an Int32 and a Double: timestamp / 1000 has type Double, assuming timestamp is of type Double (this has the same semantics as the expression timestamp / Double(1000))

Aggregates¶

Expressions in UEL can also contain aggregates. Aggregates are summary statistics maintained at different temporal granularities. They are specified using their names, zero or more group by attributes (coming from the tuple being processed), period and the window unit.

The available periods are Current, Last, and AllTime. When the period is Current, the available window units are: Day, Hour, Month and Year. If the period is Last, the Minute can also be used as the window unit. The computed aggregates are always of type Double.

The following examples illustrate the use of aggregates as part of expressions:

This aggregate, named numCallsMade, returns the number of calls made for a given number within the last hour, where callingNumber is an attribute available from the current tuple: aggregate(numCallsMade[callingNumber], Last, 1, Hour).
Aggregates can be involved in arithmetic operations as usual: aggregate(numCallsMade[callingNumber], Last, 1, Hour) + aggregate(numCallsMade[calledNumber], Last, 1, Hour)
Some aggregates may have no group by attributes: aggregate(totalCalls, Current, Month)
Some aggregates may have multiple group by attributes: aggregate(numCallsMade[callingNumber, callingCellTower], Last, 1, Hour)

Aggregates can also specify the number of most recent time units to be used for the aggregation. By default an hourly aggregate is computed from 6 10-minute aggregates, a daily aggregate is computed from 24 hourly aggregates, a monthly aggregate is computed from daily aggregates within a month, and a yearly aggregate is computed from 12 monthly aggregates. One can specify a second argument as part of the aggregate’s temporal access function, which represents the number of time units to be used for the aggregation. The time units used are always the most recent ones. For instance:

The following aggregate gets the number of calls made during the last week (7 days): aggregate(numCallsMade[callingNumber], Last, 7, Day)

The aggregates with the Current and AllTime periods provide exact results:

aggregate(<aggregate>, Current, Hour): an exact aggregate value over all the activities within the current hour. E.g.: If the current time is 14:20pm, then the activities within the last 20 minutes are included.
aggregate(<aggregate>, Current, Day): an exact aggregate value over all the activities within the current day. E.g.: If the current time is 14:20pm, then the activities since midnight are included.
aggregate(<aggregate>, Current, Month): an exact aggregate value over all the activities with the current month. E.g.: If the current time is 14 May 14:20pm, then the activities since the beginning of May up to 14:20pm on May 14th are included.
aggregate(<aggregate>, Current, Year): an exact aggregate value over all the activities with the current year. E.g.: If the current time is 14 May 2017 14:20pm, then the activities since the beginning of 2017 up to 14:20pm on May 14th are included.
aggregate(<aggregate>, AllTime): an exact aggregate value over all the activities, irrespective of time.

There are 4 possible ways of computing aggregates with the Last period:

Aggregate over the last hour: an approximate aggregate value over the activities within the last 60 minutes, that is the last 6 10-minute periods. It is an approximate value in the sense that if the current 10-minute interval is at least half past, then the aggregate is over the activities within the current 10-minute interval plus the last 5 10-minute intervals. If the current 10 minute interval is less than half past, then the aggregate is over the activities within the current 10-minute interval plus the last 6 10-minute intervals. E.g.: If the current time is 14:29pm, then the activities within the interval [13:30pm - 14:29pm] are included. If the current time is 14:21pm, then the activities within the interval [13:20pm - 14:21pm] are included.
Aggregate over the last day: an approximate aggregate value over the activities within the last 24 hours. It is an approximate value in the sense that if the current hour is at least half past, then the aggregate is over the activities within the current hour plus the last 23 calendar hours. If the current hour is less than half past, then the aggregate is over the activities within the current hour plus the last 24 calendar hours. E.g.: If the current time is Tuesday 14:50pm, then the activities within the interval [Monday 15:00pm - Tuesday 14:50pm] are included. If the current time is Tuesday 14:10pm, then the activities within the interval [Monday 14:00pm - Tuesday 14:10pm] are included.
Aggregate over the last month: an approximate aggregate value over the activities within the last 30 days. It is an approximate value in the sense that if the current day is at least half past, then the aggregate is over the activities within the current day plus the last 29 calendar days. If the current day is less than half past, then the aggregate is over the activities within the current day plus the last 30 calendar days. E.g.: If the current time is 14 May 22:00pm, then the activities within the interval [15 April 00:00am - 14 May 22:00pm] are included. If the current time is 14 May 02:00am, then the activities within the interval [14 April 00:00am - 14 May 02:00am] are included.
Aggregate over the last year: an approximate aggregate value over the activities within the last 12 months. It is an approximate value in the sense that if the current month is at least half past, then the aggregate is over the activities within the current month plus the last 11 calendar months. If the current month is less than half past, then the aggregate is over the activities within the current month plus the last 12 calendar months. E.g.: If the current time is 28 May 2017 14:00pm, then the activities within the interval [1 June 00:00am - 28 May 14:00pm] are included. If the current time is 2 May 14:00pm, then the activities within the interval [1 May 00:00am - 2 May 14:00pm] are included.

The same kind of approximation applies if the number of most recent time units are specified while accessing an aggregate. For instance, if the last 4 days are requested from the last month, then the current day plus the last 3 or 4 calendar days are included in the result, depending on whether the current day is at least half past or not, respectively.

These computations are done by using the following aggregate expressions:

aggregate(<aggregate>, Last, <windowLength>, Minute): this computes the aggregation by using windowLength minutes from the last hour.
aggregate(<aggregate>, Last, <windowLength>, Hour): if windowLength is greater than 1, this computes the aggregation over the last windowLength hours from the last day. Otherwise it computes the aggregation using the last hour.
aggregate(<aggregate>, Last, <windowLength>, Day): if windowLength is greater than 1, this computes the aggregation over the last windowLength days from the last month. Otherwise it computes the aggregation over the last day.
aggregate(<aggregate>, Last, <windowLength>, Month): if windowLength is greater than 1, this computes the aggregation over the last windowLength months from the last year. Otherwise it computes the aggregation over the last month.
aggregate(<aggregate>, Last, <windowLength>, Year): this computes the aggregation over the last year and the windowLength must be equal to 1.

Null Values¶

Attributes in UEL are nullable, that is, attributes can take null values. To denote a null value, the null keyword is used.

Only the following actions are legal on the expressions with null values:

Built-in function calls: A nullable function parameter can take a null value. E.g.: the second parameter in the list.indicesOf([1, 2, null, 4, 5, null], null) call
Ternary operations: The values returned from choices can be null. E.g.: string.startsWith(profile.areaCode, "AREA_") ? profile.areaCode : null where areaCode is a profile attribute of the String type
List items: Null values can be list items. E.g.: [null, 3, 5, 6, null]
List containment check operations: Null values can be used on the left hand side. E.g.: null not in [null, 3, 5, 6, null]
Equality check operations: Null values can be compared for equality and non-equality. E.g.: MSISDN == null where MSISDN is a tuple attribute (Note: For comparisons with null values, the isNull operator can also be used. Examples: isNull(null) yields true isNull(profile.x) yields false if the x profile attribute is not null)
Type conversions: The type conversion rules are similar to the ones for non-null expressions. Some examples that are legal: List(Int32)(null), Int32(null), String(Int32(null)), some examples that are illegal: List(Int32)(List(Int64)(null)), Int64(List(Int32)(null)), Bool(Int32(null)), List(Int16)(Double(null))

Built-in functions¶

UEL Builtin Functions