Skip to content

kdb products

Pipeline UI quickstart

kdb products

Home
kdb+ and q
kdb Insights
kdb Insights
- About
- Free Trial
- Core
  Core
  - About
  - Install
  - Quick start
    Quick start
    
    Overview
    
    Object storage
    
    REST
    REST
    
    Client
    
    Server
    
    Google Cloud Platform
    Google Cloud Platform
    
    Big Query
    
    Logging
    
    Packaging
  - Object storage
    Object storage
    
    About
    
    Caching
    
    Examples
  - SQL
    SQL
    
    About
    
    SQL Reference
    SQL Reference
    
    Operators
    
    Functions
    
    Data and Literals
    
    Select Statements
    
    Table Creation
    
    ANSI SQL Compliance
  - Postgres SQL Interface
  - REST API
    REST API
    
    Client
    Client
    
    About
    
    Workflows
    
    Examples
    Examples
    
    Async
    
    Follow redirects
    
    Response headers
    
    Timeouts
    
    Azure API Management
    
    GCP Identity Aware Proxy
    
    Server
    Server
    
    About
    
    API reference
    
    Examples
    Examples
    
    customers
    
    queryclient
    
    queryserver
    
    queryworker
    
    OpenAPI Sample
  - Google BigQuery API
    Google BigQuery API
    
    About
    
    Main
    
    Discovery
    
    Query
    
    Projects
    
    Datasets
    
    Tables
    
    Tabledata
    
    Helpers
    
    Configuration
    
    API
    
    Troubleshooting
  - Packaging
    Packaging
    
    About
    
    Examples
    Examples
    
    About the examples
    
    Basic Tick
    
    Hello C
    
    Labelling
  - Logging
    Logging
    
    About
    
    API reference
  - Release notes
    Release notes
    
    Latest
    
    Previous
- PyKX
- Microservices
  Microservices
  - About
  - Getting Started
    Getting Started
    
    Prerequisites
  - Service Discovery
    Service Discovery
    
    About
    
    Prerequisites
    
    Running
    
    Configuration
    
    OpenAPI
  - Database
    Database
    
    Overview
    
    Data configuration
    Data configuration
    
    Overview
    
    Routing
    
    Assembly
    Assembly
    
    Database
    
    Schema
    
    Storage
    
    Query
    
    Stream
    
    Aggregation
    
    Advanced
    Advanced
    
    Overview
    
    Query scaling
    
    Data storage
    Data storage
    
    Overview
    
    Storage tiering
    
    Initial import
    
    Batch ingest
    
    Data query
    Data query
    
    Overview
    
    Purviews
    
    Late data
    
    Reference data
    
    Routing
    
    Queueing, retries, and timeouts
    
    Resilience
    
    Logging
    
    Troubleshooting
    
    Advanced
    Advanced
    
    Query existing object storage
    
    Querying methods
    Querying methods
    
    REST vs QIPC
    
    SQL
    
    Monitoring
    
    Best practices
    Best practices
    
    Late Data
    
    Performance
    
    Deploying
    
    Glossary
  - Stream Processor
    Stream Processor
    
    About Streaming Data
    
    Quickstart
    Quickstart
    
    Docker
    
    Kubernetes
    
    Writing
    
    Running
    
    Configuration
    
    Examples
    Examples
    
    Static file
    
    Batch S3 ingestion
    
    Streaming Kafka ingestion
    
    Kafka with TLS
    
    PostgreSQL Querying
    
    Stateful operators
    
    Enriching streams
    
    Windowing on event time
    
    Windowing on processing time
    
    kdb+ tick (callback)
    
    Concepts
    Concepts
    
    Checkpoints and recovery
    
    Scaling
    
    State
    
    Release Notes
  - Machine Learning
    Machine Learning
    
    About
    
    Quickstart
    Quickstart
    
    Docker
    
    Kubernetes
    
    Examples
    Examples
    
    Model Generation & Deployment
  - Reliable Transport
    Reliable Transport
    
    About
    
    Quickstart
    Quickstart
    
    Docker
    
    Kubernetes
    
    Publishers
    Publishers
    
    Overview
    
    RT Bridge
    
    Subscribers
    
    SDKs and QPKs
    SDKs and QPKs
    
    Getting started
    
    C SDK
    C SDK
    
    Using the C SDK
    
    C SDK samples
    
    Java SDK
    Java SDK
    
    Using the Java SDK
    
    Java samples
    
    RT.qpk
    
    Configuration
    Configuration
    
    Overview
    
    Diagnostics
    
    Monitoring
    
    Administration
    Administration
    
    Soft reset
    
    Hard reset
  - Release notes
    Release notes
    
    Latest
    
    Previous
kdb Insights Enterprise
kdb Insights Enterprise
- Home
- About
- Architecture
- Install
  Install
  - Overview
  - Free Trial
  - Azure Marketplace
    Azure Marketplace
    
    About
    
    Installing
    Installing
    
    Prerequisites
    Prerequisites
    
    Prerequisites
    
    Permissions
    
    User Node Pool Sizing
    
    Install
    
    Login
    
    Potential issues
    
    Azure Integrations
    Azure Integrations
    
    Azure Active Directory
    Azure Active Directory
    
    Azure Active Directory
    
    AAD Keycloak Composite Roles
    
    Azure Monitoring
    Azure Monitoring
    
    Alert Configuration
    
    Workbook Configuration
    
    Azure PowerBI
    
    Billing
    Billing
    
    About
    
    FAQ
    
    Security
    
    Support
    Support
    
    KX Support
    
    Licensing
    
    Azure Secrets
    
    Release Notes
  - Standalone
    Standalone
    
    Infrastructure
    Infrastructure
    
    Prerequisites
    
    Terraform deployment
    Terraform deployment
    
    Overview
    
    Cloud provider
    Cloud provider
    
    GCP
    
    AWS
    
    Azure
    
    Interacting with Kubernetes
    
    DNS setup
    
    Installation
    Installation
    
    Installing
    
    Upgrading
- Use
  Use
  - Get started
    Get started
    
    Overview
    
    Log in
    
    User interface
    
    Product Tour
    
    Guided walkthrough
    Guided walkthrough
    
    About
    
    Ingest and Query
    Ingest and Query
    
    Database
    
    Object Storage (Weather)
    
    Kafka (Subway)
    
    SQL Database (Health)
    
    Protocol Buffer (Crime)
    
    Query
    
    Visualize
    Visualize
    
    Build a View
    
    Maps
    
    Troubleshooting
    
    Industry tutorials
    Industry tutorials
    
    Index
    
    Finance
    
    Kafka
    
    Manufacturing
  - Configuring a database
    Configuring a database
    
    Overview
    
    Configuration options
    Configuration options
    
    Overview
    
    Routing
    
    User interface
    User interface
    
    Database
    
    Schema
    
    Stream
    
    Assembly
    Assembly
    
    Database
    
    Schema
    
    Storage
    
    Query
    
    Stream
    
    Aggregation
    
    Advanced
    Advanced
    
    Overview
    
    Query scaling
    
    Deploying
    
    Monitoring
    
    Best practices
    Best practices
    
    Late data
    
    Performance
    
    Glossary
  - Storing Data
    Storing Data
    
    Overview
    
    Storage tiering
    
    Initial import
    
    Batch ingest
  - Ingest & transform
    Ingest & transform
    
    Overview
    
    Import wizard
    
    Pipelines
    Pipelines
    
    Overview
    
    Test
    
    Deploy
    
    Operators
    Operators
    
    Overview
    
    Readers
    
    Writers
    
    Functions
    
    Decoders
    
    Encoders
    
    Transform
    
    Windows
    
    Machine Learning
    
    Examples
    Examples
    
    Overview
    
    Kafka
    Kafka
    
    Getting started
    
    Setup Kafka
    
    Basic ingestion
    
    Setup Kafka TLS
    
    Enabling TLS ingestion
    
    Fitting Machine Learning model on Kafka data
    
    PostgreSQL query
    PostgreSQL query
    
    Getting started
    
    Setup PostgreSQL
    
    Querying PostgreSQL
    
    S3
    S3
    
    Batch S3 ingest
    
    Fitting Machine Learning model on S3 data
    
    Using the SDKs
  - Querying data
    Querying data
    
    Overview
    
    Purviews
    
    Late data
    
    Reference data
    
    Routing
    
    Queueing, retries and timeouts
    
    Query methods
    Query methods
    
    UI
    
    REST vs QPIC
    
    SQL
    
    Java SDK
    
    Resilience
    
    Logging
    
    Troubleshooting
    
    Advanced
    Advanced
    
    Querying existing object storage
  - Adding custom code
    Adding custom code
    
    Overview
    
    Quickstart
    
    Creating a package
    
    Uploading a package
    
    Using a package
  - Analysing your data
    Analysing your data
    
    Overview
    
    Quickstart
    
    Data exploration
    Data exploration
    
    Using the UI Explore window
    Using the UI Explore window
    
    Querying databases
    
    Developing using q
    
    Developing using Python
  - Visualizing Data
    Visualizing Data
    
    Overview
    
    Quickstart
    
    Building reports and dashboards
  - Troubleshooting
    Troubleshooting
    
    Diagnosing deployments
- Administer
  Administer
  - Command line interface
    Command line interface
    
    Overview
    
    Installing the CLI
    
    Configuration
    
    Reference
  - Assembly Deployment
    Assembly Deployment
    
    Overview
    
    Building
    
    Deploying
  - SDKs
    SDKs
    
    Getting started
    
    C SDK
    C SDK
    
    Using the C SDK
    
    C SDK Samples
    
    Java SDK
    Java SDK
    
    Using the Java SDK
    
    Java Samples
    
    RT Bridge
  - Security and authentication
    Security and authentication
    
    Authentication and authorization
    
    Shared Keycloak instance
    
    Keycloak backup and restore
  - Configuration
    Configuration
    
    Overview
    
    Setup
    
    Security
    
    Resources
    
    Availability
    
    Observability
    
    Storage
    
    RT archival
    
    Database
    
    Stream Processor
    
    Advanced
    Advanced
    
    Password policy
    
    Overprovisioning
  - Observability
    Observability
    
    Overview
    
    Logging
    
    Metrics
    
    Metrics reference
- Develop
  Develop
- Glossary
- Release notes
  Release notes
  - Latest
  - Previous
APIs
APIs
- Overview
- OpenAPI
  OpenAPI
- Packages
  Packages
  - Overview
  - Package Contents
  - Command Line Interface
  - q Interface
    q Interface
    
    Overview
    
    Packages
    
    User-Defined Functions
  - Python Interface
    Python Interface
    
    Overview
    
    Packages
    
    User Defined Functions
- Database
  Database
  - Overview
  - Interface
    Interface
    
    Overview
    
    Header
    
    Codes
  - Query
    Query
    
    Overview
    
    Get Data
    
    Get Meta
    
    Ping
    
    QSQL
    
    SQL
  - Custom APIs
    Custom APIs
    
    Overview
    
    Metadata
    
    Registration
  - OpenAPI
    OpenAPI
    
    Overview
    
    Service Gateway
    
    Resource Coordinator
    
    Aggregator
    
    Data Access
    
    Storage Manager
- Stream Processor
  Stream Processor
  - Overview
  - q Interface
    q Interface
    
    Overview
    
    Configuring Operators
    
    General
    
    Lifecycle
    
    Operators
    
    Data Structures
    
    Readers
    
    Decoders
    
    Encoders
    
    Data Transforms
    
    Stats
    
    State
    
    Windows
    
    Writers
    
    User-Defined Functions
    
    Machine Learning
  - Python Interface
    Python Interface
    
    Overview
    
    General
    
    Lifecycle
    
    Operators
    
    Readers
    
    Decoders
    
    Encoders
    
    Data Transforms
    
    State
    
    Windows
    
    Writers
    
    Machine Learning
  - OpenAPI
    OpenAPI
    
    Coordinator
    
    Controller
    
    Worker
- Machine Learning
  Machine Learning
  - About
  - q Interface
    q Interface
    
    About
    
    Analytics
    Analytics
    
    About
    
    ML Analytics API
    ML Analytics API
    
    Introduction
    
    ML Toolkit
    
    Online Models
    Online Models
    
    Introduction
    
    Stochastic Gradient Descent
    Stochastic Gradient Descent
    
    Stochastic Gradient Descent
    
    Linear Regression
    
    Logistic Classification
    
    Secure Updates
    
    Sequential K Means
    
    Variadic Functionality
    Variadic Functionality
    
    Introduction
    
    Function Calls
    
    Clustering models
    
    Statistical models
    
    Time series models
    
    Online models
    
    Registry
    Registry
    
    About
    
    Cloud Integration
    
    Registry API
    Registry API
    
    Storing
    
    Loading
    
    Deleting
    
    Examples
    Examples
    
    Basic Examples
  - Python Interface
    Python Interface
    
    About
    
    Registry
    Registry
    
    About
    
    Cloud Integration
    
    Registry API
    Registry API
    
    Storing
    
    Loading
    
    Deleting
    
    Examples
    Examples
    
    Basic Python Examples
Licensing
Licensing
- About
- Installation
- Quick start
- Licensing workflow
- Service accounts
- Usage data
- Access control
- Walkthroughs
  Walkthroughs
  - Containers
  - Kubernetes
- Appendix
  Appendix
  - Identity
  - Offline enrolment
Help

Pipeline UI quickstart

An example of building and deploying a pipeline in the Insights user interface