Python Objects¶

This page lists python objects that are referenced from elsewhere in the documentation.

Pipeline Objects¶

class Pipeline()

Thin Python wrapper around a q Stream Processor pipeline.

This class provides special handling for the | operator. When given a Pipeline instance on either side, it produces a new Pipeline instance that is comprised of the operators from the pipeline on the left connected in series with those from the pipeline on the right.

The | operator does not modify a pipeline definition in-place. Pipelines are immutable.

When the right argument of | is a list of pipelines, and the left argument is a single pipeline, the result is a list where each element is a pipeline with the left argument joined to one of the elements from the list on the right.

When the left argument of | is a list of pipelines, and the right argument is a single pipeline, the result is a single pipeline obtained by taking the union of every pipeline from the list with the pipeline on the right.

Examples:

Join a reader to a map operation, then join that pipeline to a writer, and run the resulting pipeline:

>>> from kxi import sp

>>> sp.run(sp.read.from_expr(lambda: range(10, 30, 2))
    | sp.map(lambda x: x*x)
    | sp.write.to_console(timestamp='none'))

100 144 196 256 324 400 484 576 676 784

Join a reader to multiple map operations, and then have them all output to the same writer:

>>> from kxi import sp

>>> reader = sp.read.from_expr(lambda: range(10))
>>> maps = (sp.map(lambda x: x), sp.map(lambda x: x ** 2), sp.map(lambda x: x ** 3))
>>> writer = sp.write.to_console(timestamp='none')
>>> sp.run(reader | maps | writer)

0 1 2 3 4 5 6 7 8 9
0 1 4 9 16 25 36 49 64 81
0 1 8 27 64 125 216 343 512 729

as_dot¶

@cached_property
def as_dot() -> str

Provides the graph structure of the pipeline in the DOT format.

validate¶

def validate() -> None

Validate the structure of the pipeline graph.

Raises:

pykx.QError - If the pipeline is not valid; the error message will explain why.

SPModule Objects¶

class SPModule(ModuleType)

mode¶

@cached_property
def mode()

The mode of the Stream Processor.

'local' if operating independently as a local deployment.
'cluster' if operating as part of a larger cluster with a parent controller.

OperatorParams Objects¶

class OperatorParams(AutoNameEnum)

Specifies a parameter that will be provided to the function you provide to an operator.

operator¶

Provide the operator's dictionary, which may be required by other SP functions.

metadata¶

Provide the message's metadata, which may be required by other SP functions.

data¶

Provide the message's data.

Operator Objects¶

class Operator()

Stream Processor operator interface.

An operator is a first-class building block in the stream processor API. Operators can be strung together to form a kxi.sp.Pipeline instance, which can then be run.

Pipeline and operator objects can be joined together using the | operator. Operators can also be joined to lists/tuples of pipelines or operators.

See Also:

kxi.sp.Pipeline

Attributes:

id str - The unique ID of this operator.