Stream Processor Metadata¶
Stream Processor metadata is exposed across many APIs, such as state and pushing data further in a pipeline within asynchronous nodes. A pipeline operator's metadata is a dictionary containing the following keys. These keys will be extended as new capabilities are added to the Stream Processor engine.
|key||any||An optional partition key for the events in this message. This is used to group related collections of data based on a common attribute. Events that do not share the same key are considered independent and may be processed independent of other keys.|
|window||temporal||Indicates the start time or index of a window of data.|
|offset||int or temporal||An event offset is a number or time that represents the current message offset to the stream reader. This value will be used to recover data if messages are lost down stream. In the case messages need to be replayed, the message offset will be used to indicate the last successfully processed message.|
Publishes data to all downstream operators in the pipeline.
.qsp.push should be
used from asynchronous operators in a pipeline to continue the flow of data in the pipeline.
It is only required for
.qsp.apply. Other operators are synchronous and any data returned
will flow to the next operator.
|op||#.sp.op||The current operator configuration. To access the operator configuration from the body of an operator function, use
|md||#.qsp.metadata||Metadata for message that is being emitted from the current asynchronous operator.|
|data||any||Data that is to be published from this async operator.|
|any||Returns the data that was passed to this function.|
See Also: .qsp.use
Installs and runs a pipeline in the current stream processor. Depending on the deployment configuration, this function can run a pipeline in the current worker process or distribute work across several workers.
Tear down a pipeline, removing all state and timers.
Any 'onTeardown' (using
.qsp.onTeardown) handlers will be called.