Skip to content

Configuring Operators

.qsp.use

Modifies the behavior of an operator by providing custom configuration options to the operator. .qsp.use can be passed as an argument to any operator to modify the operator's behavior.

Parameters:

Name Type Description
opts dict Custom configuration options.
opts.symbol any

Returns:

Type Description
dict Custom configuration options formatted to be passed to an operator.

Example: Making a node stateful and named

  // A function that takes an operator and the current metadata as the first
  // two arguments, and the data as a third argument
  add: {[op; md; data] .qsp.set[op; md] .qsp.get[op; md] + data };

  // Create a pipeline that creates a `publish` callback, and sends data
  // to a stateful map. The `map` is configured with a specific name
  // (overriding the generated name), and is configured to be stateful
  // with a default state of `0`. Specifying an operation as stateful
  // adds the required `op` (operator) and `md` (metadata) arguments
  // to the user defined function.
  .qsp.read.fromCallback[`publish]
      .qsp.map[add; .qsp.use `name`state!(`MyStatefulApply; 0)]

  // Since the name was set explicitly for the map node, the state held in the
  // node can be queried externally through q IPC or the REST API:
  // `:worker:5000 (`.qsp.get; `MyStatefulApply; ::)
  // curl worker:5000/state/MyStatefulApply

Example: Retrieving node metadata

 // The metadata for a node can be retrieved with the `params` option:
 addWindowStart: {[md; data] update start: md`window from data }

 // The metadata added by the window operator can be accessed by the map
 // by specifying the `params` field, adding the `md` (metadata) argument
 // to the user defined function.
 .qsp.read.fromCallback[`publish]
     .qsp.window.tumbling[00:00:01; {y`timestamp}]
     .qsp.map[addWindowStart; .qsp.use``params!(::; `metadata`data)]

Example: Add node-custom configuration

 // In addition to general node configuration, each operator may specify a set
 // of optional configurations that can be set using `.qsp.use` as well. For example,
 // window operations allow a `snap` configuration to align windows. By default this
 // is disabled, but can be enabled explicitly:
 .qsp.read.fromCallback[`publish]
     .qsp.window.tumbling[00:00:10; {y`timestamp}; .qsp.use``snap!(:: 1b)]