Home > Hadrian >

This page provides an introduction to using Titus. It deliberately mirrors the Hadrian Basic Use page.

Before you begin…

Download and install Titus. This article was tested with Titus 0.8.3; newer versions should work with no modification. Python >= 2.6 and < 3.0 is required.

Launch a Python prompt and import json and PFAEngine:

Python 2.7.6
Type "help", "copyright", "credits" or "license" for more information.
>>> import json
>>> from titus.genpy import PFAEngine

Simplest possible scoring engines

Let’s start with an engine that merely adds 10 to each input. That’s something we can write inline.

>>> engine, = PFAEngine.fromJson('''
... {"input": "double",
...  "output": "double",
...  "action": {"+": ["input", 10]}}
... ''')

For convenience, we could have written it in YAML (all of Titus’s unit tests are written this way).

>>> engine, = PFAEngine.fromYaml('''
... input: double
... output: double
... action: {"+": [input, 100]}
... ''')

Notice the comma (,) after engine. The PFAEngine.fromJson and PFAEngine.fromYaml functions produce a collection of PFAEngine objects from one PFA file (pass multiplicity = 4 and drop the comma to see that). The comma makes the left-hand side a one-element tuple, effectively unpacking the singleton list. These scoring engines can run in parallel and share memory. For now, though, we’re only interested in one scoring engine.

By virtue of having created an engine, the PFA has been fully validated. If the PFA is not valid, you would see

Now run the scoring engine on some sample input:

>>> print engine.action(3.14)
103.14

You should only ever see one of the following exceptions at runtime

Emit-type engines

Of the three types of PFA scoring engine (map, emit, and fold), emit requires special attention in scoring. Map and fold engines yield results as the return value of the function (and fold do so cumulatively), but emit engines always return None. The only way to get results from them is by passing a callback function.

>>> engine2, = PFAEngine.fromYaml('''
... input: double
... output: double
... method: emit
... action:
...   - if:
...       ==: [{"%": [input, 2]}, 0]
...     then:
...       - emit: input
...       - emit: {/: [input, 2]}
... ''')
... 
>>> def newEmit(x):
...     print "output:", x
... 
>>> engine2.emit = newEmit
>>> 
>>> for x in range(1, 5+1):
...     print "input:", x
...     engine2.action(x)
input: 1
input: 2
output: 2.0
output: 1.0
input: 3
input: 4
output: 4.0
output: 2.0
input: 5

Titus Data Format

Data passed to Titus (or received from Titus) must take the following form.

Avro type Type in Titus Example
null NoneType None
boolean bool True, False
int int or long 3
long int or long 3L
float int, long, or float 3.14
double int, long, or float 3.14
string str or unicode (Python 2) "hello"
bytes str (Python 2) "\x00\x01\x02"
array list or tuple [1, 2, 3]
map dict {"one": 1, "two": 2}
record dict {"x": 1, "y": "hello"}
fixed str (Python 2) "\x00\x01\x02"
enum str or unicode (Python 2) "third"
union untagged object 3, "hello", or None
  tagged object {"int": 3}, {"string": "hello"}, or None

Titus functions are designed to accept unions in tagged or untagged form and produce unions in tagged form. The Python Avro library produces tagged unions and unicode strings and the fastavro library produces untagged unions and raw strings.

Snapshots

Snapshots are representations of a PFA engine’s state at a moment in time. They are only relevant if the engine has a mutable state. Let’s start by making a mutable scoring engine and filling it with some state.

>>> engine4, = PFAEngine.fromYaml('''
... input: int
... output: {type: array, items: int}
... cells:
...   history:
...     type: {type: array, items: int}
...     init: []
... action:
...   cell: history
...   to: {a.append: [{cell: history}, input]}
... ''')
... 
>>> engine4.action(1)
[1]
>>> engine4.action(2)
[1, 2]
>>> engine4.action(3)
[1, 2, 3]
>>> engine4.action(4)
[1, 2, 3, 4]
>>> engine4.action(5)
[1, 2, 3, 4, 5]

The snapshot method locks the scoring engine and turns the state of the engine into a new AST that could be immediately serializxed as a PFA file.

>>> engine4.snapshot()
EngineConfig(name=Engine_3,
    method=map,
    inputPlaceholder="int",
    outputPlaceholder={"items": "int", "type": "array"},
    begin=[],
    action=[CellTo(u'history', [], Call(u'a.append', [CellGet(u'history', []), Ref(u'input')]))],
    end=[],
    fcns={},
    zero=None,
    merge=None,
    cells={u'history': Cell({"items": "int", "type": "array"}, '[1, 2, 3, 4, 5]', False, False, 'embedded')},
    pools={},
    randseed=None,
    doc=None,
    version=None,
    metadata={},
    options={})

To get the values, dig into the EngineConfig object to get the init of the relevant cell or pool. Then use json.loads to convert the serialized form into an object.

>>> json.loads(engine4.snapshot().cells["history"].init)
[1, 2, 3, 4, 5]

Abstract Syntax Tree

The PFA AST is an immutable tree structure built from the serialized JSON, stored in engine.config, which is an EngineConfig. You can query anything about the original PFA file in a structured way through this AST. For instance,

>>> engine.config.action[0]
Call(u'+', [Ref(u'input'), LiteralInt(100)])

>>> engine.config.action[0].__class__.__name__
'Call'

>>> engine.config.input.avroType
"double"

There are also a few methods for recursively walking over the AST. The collect method applies a partial function to all nodes in the tree and produces a list of matches. For instance, to get all Expressions (function calls like “+”, symbol references like “input”, and literal values like “100”), do

>>> from titus.pfaast import Expression
>>> def pf(x): return x
... 
>>> pf.isDefinedAt = lambda x: isinstance(x, Expression)
>>> 
>>> engine.config.collect(pf)
[Call(u'+', [Ref(u'input'), LiteralInt(100)]), Ref(u'input'), LiteralInt(100)]

The function object (pf in this case) must have another function associated with it to define the domain, making it a partial function in analogy with Scala’s PartialFunction class.

You can also build new scoring engines by passing a replacement function. This one turns instances of 100 into 999. You can do quite a lot just by crafting the right partial function.

>>> from titus.pfaast import LiteralInt
>>> def pf(x): return LiteralInt(999)
... 
>>> pf.isDefinedAt = lambda x: isinstance(x, LiteralInt) and x.value == 100
>>> 
>>> engine.config.replace(pf)
EngineConfig(name=Engine_1,
    method=map,
    inputPlaceholder="double",
    outputPlaceholder="double",
    begin=[],
    action=[Call(u'+', [Ref(u'input'), LiteralInt(999)])],
    end=[],
    fcns={},
    zero=None,
    merge=None,
    cells={},
    pools={},
    randseed=None,
    doc=None,
    version=None,
    metadata={},
    options={})

In fact, this is how Titus generates code in general. A walk over the tree checks for semantic errors while calling a Task at each node. Usually, this Task is to create Python code, but it could be anything. This small example generates Lisp.

>>> from titus.pfaast import *
>>> from titus.datatype import *
>>> from titus.options import EngineOptions
>>> from titus.signature import PFAVersion
>>> 
>>> class LispCode(TaskResult): pass
... 
>>> class LispFunction(LispCode):
...     def __init__(self, car, cdr):
...         self.car = car
...         self.cdr = cdr
...     def __repr__(self):
...         return "(" + self.car + " " + " ".join(repr(x) for x in self.cdr) + ")"
... 
>>> class LispSymbol(LispCode):
...     def __init__(self, name):
...         self.name = name
...     def __repr__(self):
...         return self.name
... 
>>> class GenerateLisp(Task):
...     def __call__(self, context, engineOptions):
...         if isinstance(context, Call.Context):
...             return LispFunction(context.fcn.name, context.args)
...         elif isinstance(context, Ref.Context):
...             return LispSymbol(context.name)
...         elif isinstance(context, LiteralInt.Context):
...             return LispSymbol(str(context.value))
... 
>>> symbolTable = SymbolTable(None, {}, {}, {}, True, False)
>>> symbolTable.put("input", AvroDouble())
>>> engine.config.action[0].walk(GenerateLisp(), symbolTable, FunctionTable.blank(), \
...                              EngineOptions({}, {}), PFAVersion(0, 8, 1))[1]
... 
(+ input 100)
>>> 
>>> engine6, = PFAEngine.fromYaml('''
... input: double
... output: double
... action: {+: [{/: [input, 2]}, {m.sqrt: input}]}
... ''')
... 
>>> engine6.config.action[0].walk(GenerateLisp(), symbolTable, FunctionTable.blank(), \
...                               EngineOptions({}, {}), PFAVersion(0, 8, 1))[1]
... 
(+ (/ input 2) (m.sqrt input))