Skip to content

Pipeline

Pipelines objects are constructed by the CLI from config files in the pipelines/ directory.

In a package directory, each pipeline is typically stored as {package-root}/pipelines/{pipeline-name}.yaml and referenced from the package Manifest.

Each pipeline either contains its spec inline or references a source file in the package that contains the logic used to run the pipeline.

By default the CLI creates pipeline objects and a corresponding src file at src/{pipeline-name}.q. On the UI these same pipelines are initialized with an json object graph spec.

Key Value
type object
relationship Package -> Pipeline
object path $.pipelines[*]
package file path {package-root}/pipelines/{pipeline-name}.yaml
cli commands kxi package add --to $PKG pipeline --name $PIPELINE
kxi package rm --from $PKG pipeline --name $PIPELINE

Fields

Field Type Required Description Constraints Default
base Base | null no Base image to deploy - null
config map<string, string > | null no Additional configuration to be applied to SP Pipeline Assembly element - null
configMaps array<ConfigMap> | null no Pre-configured Kubernetes config maps to inject into pipeline - null
controller Controller | null no Configure Pipeline Controller - null
destination string | null no Sequencer Bus to publish to 'destination' is used to look up sequencers defined within the Assembly 'elements.sequencer' or 'bus' fields. If 'destination' is not located, it is assumed to be an existing Sequencer name, external to the Assembly. pattern: "^[a-z0-9]+[a-z0-9-]*[a-z0-9]+$" null
env array<EnvItem> | null no Env provides a list of environment variable to inject into the container. See https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/ - null
group string | null no Groups a pipeline into a set of replicas that have a matching group id pattern: "^[a-z0-9]+[a-z0-9-]*[a-z0-9]+$" null
id string | null no SP Pipeline ID - null
imagePullSecrets array<ImagePullSecret> | null no Pre-configured Kubernetes imagePullSecrets to inject into pipeline - null
maxWorkers integer | null no Maximum worker instances min: 0 null
metadata ComponentMetadata | null no - - null
metrics Metrics | null no Configure pipeline metrics - null
minWorkers integer | null no Minimum worker instances min: 0 null
mode Mode | null no Pipeline mode, one of default or debug - null
monitoring boolean | null no Enable monitoring on Pipeline pods - null
name string | null no Name of the pipeline deployment pattern: "^[a-z0-9]+[a-z0-9-]*[a-z0-9]+$" null
protectedExecution boolean | null no Enable Protected Execution - null
replicaAffinityTopologyKey ReplicaAffinityTopologyKey | null no Used for affinity and anti-affinity rules related to replicas - "zone"
replicas integer | null no Number of pipeline replicas min: 1 1
secrets array<ImagePullSecret> | null no Pre-configured Kubernetes secrets to inject into pipeline - null
source string | null no Sequencer Bus to subscribe to. 'source' is used to look up sequencers defined within the Assembly 'elements.sequencer' or 'bus' fields. If 'source' is not located, it is assumed to be an existing Sequencer name, external to the Assembly. pattern: "^[a-z0-9]+[a-z0-9-]*[a-z0-9]+$" null
spec string no - - ""
type Type | null no "graph" or "spec" pipeline deployment - "spec"
unsafeMode boolean | null no Enable unsafe mode, disables determinism - null
uuid string | null no - format: "uuid" -
volumes array<Volume1> | null no List of volumes to attach to Pipeline - null
worker Worker | null no Configure Pipeline Worker - null

Used In

Back to top