kdb Insights

Workbook documentation

kdb Insights

Home
kdb Insights SDK
kdb Insights SDK
- About
- Free Trial
- Prerequisites
- Core
  Core
  - About
  - Install
  - Object storage
    Object storage
    
    About
    
    Quickstart
    
    Caching
    
    Examples
  - SQL
    SQL
    
    About
    
    SQL Reference
    SQL Reference
    
    Operators
    
    Functions
    
    Data and Literals
    
    Select Statements
    
    Table Creation
    
    ANSI SQL Compliance
  - Postgres SQL Interface
  - REST API
    REST API
    
    Client
    Client
    
    About
    
    Quickstart
    
    Workflows
    
    Examples
    Examples
    
    Async
    
    Follow redirects
    
    Response headers
    
    Timeouts
    
    Azure API Management
    
    GCP Identity Aware Proxy
    
    Server
    Server
    
    About
    
    Quickstart
    
    API reference
    
    Examples
    Examples
    
    customers
    
    queryclient
    
    queryserver
    
    queryworker
    
    OpenAPI Sample
  - Google BigQuery API
    Google BigQuery API
    
    About
    
    Quickstart
    
    Main
    
    Discovery
    
    Query
    
    Projects
    
    Datasets
    
    Tables
    
    Tabledata
    
    Helpers
    
    Configuration
    
    API
    
    Troubleshooting
  - Packaging
    Packaging
    
    About
    
    Quickstart
    
    Examples
    Examples
    
    About the examples
    
    Basic Tick
    
    Hello C
    
    Labeling
  - Logging
    Logging
    
    About
    
    Quickstart
    
    API reference
  - Release notes
    Release notes
    
    Latest
    
    Previous
- Database
  Database
  - Overview
  - Data Configuration
    Data Configuration
    
    Overview
    
    Routing
    
    Assembly
    Assembly
    
    Database
    
    Schema
    
    Storage
    
    Query
    
    Stream
    
    Aggregation
    
    User Defined Analytics
    User Defined Analytics
    
    Overview
    
    Configuration
    
    Quickstart
    
    Testing a UDA
    
    UDA Examples
    
    Troubleshooting
    
    Best Practices
    
    Advanced
    
    Advanced
    Advanced
    
    Overview
    
    Query scaling
    
    Authorization
    Authorization
    
    Custom IPC Authorization
    
    Custom HTTP Authorization
    
    Query IPC Externally
  - Data Storage
    Data Storage
    
    Overview
    
    Storage Tiering
    
    Object Storage
    
    Delete Rows
    
    Backup and Restore
    
    Event Hooks
  - Data Import
    Data Import
    
    Import Overview
    
    Initial Import
    Initial Import
    
    Overview
    
    Prerequisites
    
    Quickstart
    
    Initial Import Process
    
    Schema Creation
    
    Troubleshooting
    
    Batch Ingest
  - Data Query
    Data Query
    
    Overview
    
    Purviews
    
    Scope
    
    Late data
    
    Reference data
    
    Routing
    
    Queuing, retries, and timeouts
    
    Resilience
    
    Logging
    
    Troubleshooting
    
    Advanced
    Advanced
    
    Query existing object storage
  - Querying methods
    Querying methods
    
    Overview
    
    REST vs QIPC
    
    SQL
  - Monitoring
  - Best practices
    Best practices
    
    Late Data
    
    Manual EOD Trigger
    
    Performance
    
    Query
  - Deploying
    Deploying
    
    Overview
    
    Docker
    Docker
    
    Database
    
    Basic
    
    Metrics
    
    Kubernetes
    Kubernetes
    
    Database
    
    Gateway
  - Downgrading
  - Glossary
- Stream Processor
  Stream Processor
  - About Streaming Data
  - Quickstart
    Quickstart
    
    Docker
    
    Kubernetes
  - Writing
  - Running
  - Configuration
  - Guides
    Guides
    
    Object storage ingestion
    
    Database ingestion
    
    kdb Insights Streams
  - Examples
    Examples
    
    Static file
    
    Batch S3 ingestion
    
    Kafka
    Kafka
    
    Getting started
    
    Setup Kafka
    
    Basic ingestion
    
    Setup Kafka TLS
    
    Ingestion with TLS
    
    PostgreSQL Querying
    
    Pipeline Replicas
    
    Stateful operators
    
    Enriching streams
    
    Windowing on event time
    
    Windowing on processing time
    
    kdb+ tick (callback)
    
    Reader Triggering
  - Concepts
    Concepts
    
    Checkpoints and recovery
    
    Determinism
    
    Glob patterns
    
    Scaling
    
    Securing pipeline credentials
    
    State
- Reliable Transport
  Reliable Transport
  - About
  - Quickstart
    Quickstart
    
    About
    
    Docker
    
    Kubernetes
  - Publishers
    Publishers
    
    Overview
  - Subscribers
  - Interfaces
    Interfaces
    
    Getting started
    
    C
    C
    
    Using the C interface
    
    C samples
    
    Java
    Java
    
    Using the Java interface
    
    Java samples
    
    Python
    Python
    
    Using the Python interface
    
    Python samples
    
    q (rt.qpk)
    
    C#
    C#
    
    Using the C# interface
    
    C# samples
  - Examples
    Examples
    
    Publishing and Subscribing to Enterprise using q
    
    Recovering archived logs
    
    Running RT outside of a container
  - Configuration
    Configuration
    
    Overview
    
    Diagnostics
    
    Monitoring
  - Administration
    Administration
    
    Soft reset
    
    Hard reset
- Release notes
  Release notes
  - Latest
  - Previous
- Extras
  Extras
  - Tutorials
    Tutorials
    
    Streaming to a web-socket client
  - Machine Learning
    Machine Learning
    
    About
    
    Quickstart
    Quickstart
    
    Docker
    
    Kubernetes
    
    Examples
    Examples
    
    Model Generation & Deployment
kdb Insights Enterprise
kdb Insights Enterprise
- Home
- Deploy
  Deploy
  - Deployment Options
  - Standalone Deployments
    Standalone Deployments
    
    Infrastructure
    Infrastructure
    
    K8S Managed
    K8S Managed
    
    Prerequisites
    
    Azure User Node Pool Sizing
    
    Terraform
    Terraform
    
    Deployment Overview
    
    Deploy on GCP
    
    Deploy on AWS
    
    Deploy on Azure
    
    Upgrade Kubernetes Cluster
    
    Upgrade Third-party Dependencies
    
    Interact with Kubernetes
    
    DNS setup
    
    K8S On-Prem Prerequisites
    
    OpenShift On-Prem Prerequisites
    
    Installation
    Installation
    
    Installing
    
    Validation
    
    Air-gapped environments
- Get Started
  Get Started
  - About kdb Insights Enterprise
  - Free Trial
    Free Trial
    
    7-day Free Trial
    
    Product Tour
  - Interfaces
  - CLI
    CLI
    
    Installation
    
    Configuration
    
    Authentication
- Learn
  Learn
  - Architecture
  - Language Interfaces
  - Command Line Interface
  - kdb VS Code Extension
  - Package Overview
  - Web Interface Guide
    Web Interface Guide
    
    About
    
    Log into kdb Insights Enterprise
    
    Navigate the Web Interface
    
    System Information
    
    Databases
    Databases
    
    Create and Manage
    
    Database Settings
    
    Schema Settings
    
    Stream Settings
    
    Database Resources
    
    Deploy/Teardwon/Cleanup
    
    Pipelines
    Pipelines
    
    Import Wizard
    
    Create and Configure
    
    Settings
    
    Operators
    Operators
    
    Overview
    
    Readers
    
    Decoders
    
    Functions
    
    Transform
    
    Windows
    
    String Utilities
    
    Stats
    
    Machine Learning
    
    Encoders
    
    Writers
    
    Test Deploy
    
    Queries
    Queries
    
    Queries Index
    
    Query Window
    
    Query Panel
    
    Scratchpad
    
    Scratchpad using q
    
    Scratchpad using Python
    
    Views
    Views
    
    About
    
    Guide to Building Views
    
    View-Only User
    
    Packages
  - Store Data
    Store Data
    
    Database Overview
    
    Database Setup
    
    Database Storage
    
    Storage Tiers
    
    Best Practices
    Best Practices
    
    Late Data
    
    Performance
  - Ingest and Transform Data
    Ingest and Transform Data
    
    Import data
    
    Initial Import Overview
    
    Ingest and Transform Overview
  - Query Data
    Query Data
    
    Query Overview
    
    Purviews
    
    Scope
    
    Routing
    
    Queueing, Retries and Timeout
  - User Defined Analytics
  - Entitlements
  - KDB-X Workloads
  - Observe and Monitor
  - KX Academy Training Course
- How To
  How To
  - Configure kdb Insights Enterprise
    Configure kdb Insights Enterprise
    
    Install Configuration
    
    Base Configuration
    
    User Access
    
    Resources
    
    Availability
    
    Encryption
    Encryption
    
    Data in Transit
    
    Data at Rest
    
    Observability
    
    Storage
    
    RT Archival
    
    Advanced
    Advanced
    
    Database
    
    Scale Pods
    
    Stream Processor
    
    Overprovisioning
    
    Keycloak and PostgreSQL Config
    
    Manage Azure Secrets
  - Manage Users and Groups
    Manage Users and Groups
    
    Authentication
    
    Manage Groups
    
    Manage Service Accounts
    
    Manage Users
    
    Password Policy Text
    
    Shared Keycloak Instance
    
    Emebdding in an iFrame
  - Manage Entitlements
    Manage Entitlements
    
    Prerequisites
    
    Configure
    
    Package Entitlements
    
    Data Entitlements
    Data Entitlements
    
    Configure Data Entitlements
    
    Data Entitlements Quickstart
    
    Row-Level Entitlements
    Row-Level Entitlements
    
    Configure Row-Level Entitlements
    
    Row-Level Entitlements Quickstart
  - Work with Packages
    Work with Packages
    
    Configure Package
    
    Create Package
    
    Deployment Components
    
    Runtime Components
    
    Functions in a package
    
    Dependent and Patch Components
    
    Edit Components
    
    Upload Package
    
    Deploy Package
    
    Automated Package Deployment
    
    Use Package
    
    List Packages
    
    Load Packages
    
    Download Package
    
    Backup and Restore Package
    
    Teardown Package
    
    Delete Package
    
    Pack Package
    
    Convert Assembly to Package
  - Configure User Defined Analyics
    Configure User Defined Analyics
    
    Configuration
    
    Quickstart
    
    Testing a UDA
    
    UDA Examples
    
    Best Practices
    
    Advanced
  - Store Data
    Store Data
    
    Configure Databases with CLI
    Configure Databases with CLI
    
    Routing Configuration
    
    Package
    Package
    
    Configure Database
    
    Configure Schema
    
    Configure Storage
    
    Configure Routing
    
    Configure Query
    
    Configure Stream
    
    Package Shared Objects
    
    Aggregation
    
    Persist to Object Storage
    
    Manual EOD Trigger
    
    Delete Rows
    
    Event Hooks
  - Ingest and Transform Data
    Ingest and Transform Data
    
    Ingest and Transform Overview
    
    Initial Import
    Initial Import
    
    Prerequisites
    
    Initial Import Quickstart
    
    Initial Import Process
    
    Batch Ingest
    
    Secure Pipelines with Kubernetes Secrets
  - Query Data
    Query Data
    
    Query Methods
    Query Methods
    
    Overview
    
    REST and QIPC
    
    SQL Query Support
    
    Java Interface Query API
    
    Power Bi Connector
    
    Late Data Queries
    
    Reference Data
    
    Query Scaling
    
    Query Resilience
    
    Logging
  - Visualize Data
  - Develop with KDB-X Workloads
  - Integrations
    Integrations
    
    Microsoft Entra ID
    Microsoft Entra ID
    
    Entra Integration
    
    Composite Roles
    
    Power Bi Connector
  - Observe and Monitor
    Observe and Monitor
    
    Logging
    
    Retrieve Logs
    
    Database Monitoring Metrics
    
    Deploy Prometheus
    
    Monitoring Stack
    Monitoring Stack
    
    Deploy using CLI
    
    Package Manager Journaling
  - Backup and Restore
    Backup and Restore
    
    KXI Deployment
    
    Keycloak Data
    
    Database
- Examples
  Examples
  - Walkthroughs and Examples Index
  - Databases
    Databases
    
    Create a Database
    
    Create Schema Script
  - Ingest Data
    Ingest Data
    
    Using the Web Interface
    Using the Web Interface
    
    Ingest Object Storage Data
    
    Stream Data from Kafka
    
    Streaming Kafka Ingest
    
    Ingest SQL Data
    
    Ingest Protocol Buffers Data
    
    Ingest Data from Parquet Files
    
    Using the CLI
    Using the CLI
    
    Use Cases
    
    Kafka Ingest
    Kafka Ingest
    
    Getting started
    
    Setup Kafka
    
    Basic ingestion
    
    Setup Kafka TLS
    
    Ingestion with TLS
    
    Machine Learning Clustering
    
    Amazon S3 Ingest
    Amazon S3 Ingest
    
    Batch S3 ingestion
    
    Batch S3 Ingest and Preprocessing
    
    PostgreSQL Ingest
  - Query Ingested Data
  - View Data
    View Data
    
    View Ingested Data
    
    Add a Map to a View
    
    Visualize Streaming Data
  - Industry Examples
    Industry Examples
    
    Finance - Develop Trading Strategies
    
    Finance - Realtime ML Stock Prediction
    
    Manufacturing - Realtime ML Stock Prediction
  - Use Language Interfaces
- Reference
  Reference
  - Packaging
    Packaging
    
    Dependencies
    
    Overlays and Patches
    
    Package Shared Objects
  - Databases
    Databases
    
    Event Hooks
  - Workloads
    Workloads
    
    KDB-X Workload Yaml
  - Observability and Monitoring
    Observability and Monitoring
    
    Alerts Reference
    
    Metrics Reference
    
    Grafana Reference
  - CLI Reference
  - Glossary
- Releases
  Releases
  - kdb Insights Enterprise
    kdb Insights Enterprise
    
    Latest
    
    Previous
  - Beta Features Terms
  - Azure Marketplace
  - Upgrading
    Upgrading
    
    kdb Insights Enterprise
    
    Azure
- Help and Support
  Help and Support
  - Product Support
  - Azure License Billing
    Azure License Billing
    
    Private Offers
    
    Billing FAQ
  - Troubleshooting
    Troubleshooting
    
    Diagnostics
    
    Deploying with IaC
    
    Data Import
    
    Database and Pipeline Health
    
    Pipelines
    
    Databases
    
    User Defined Analytics
APIs
APIs
- Overview
- OpenAPI
  OpenAPI
  - Introduction
  - q client generation
- Packages
  Packages
  - Overview
  - q Interface
    q Interface
    
    Overview
    
    Packages
    
    User-Defined Functions
  - Python Interface
    Python Interface
    
    Overview
    
    Packages
    
    User Defined Functions
  - Open API
- Database
  Database
  - Overview
  - Interface
    Interface
    
    Overview
    
    Header
    
    Codes
  - Query
    Query
    
    Overview
    
    Query APIs
    
    Get Data
    
    Get Meta
    Get Meta
    
    Get Meta
    
    Get Meta v2
    
    Get Meta v3
    
    Ping
    
    QSQL
    
    SQL
    
    SQL2
    SQL2
    
    SQL2
    
    SQL2 Select Statements
    
    SQL2 Functions and Operators
    
    Preview
  - User Defined Analytics (UDAs)
    User Defined Analytics (UDAs)
    
    Helper Functions
    
    Metadata Functions
    
    Register Functions
    
    Codes
    
    Publishing
  - REST API
    REST API
    
    Overview
    
    Service Gateway
    
    Resource Coordinator
    
    Aggregator
    
    Data Access
    
    Storage Manager
- Reliable Transport
  Reliable Transport
  - Overview
  - APIs
    APIs
    
    Archiver log history
    
    Hard reset
    
    Latest output position
    
    RT clients
    
    Soft reset
    
    Deduplication publisher clients
  - OpenAPI
    OpenAPI
    
    Worker
- Stream Processor
  Stream Processor
  - Overview
  - Configuring Operators
  - General
  - Lifecycle
  - Operators
  - Readers
  - Decoders
  - Encoders
  - Transform
  - Stats
  - State
  - String Utilities
  - Windows
  - Writers
  - Machine Learning
  - User-Defined Functions
  - Object Reference
    Object Reference
    
    q
    
    Python
  - OpenAPI
    OpenAPI
    
    Coordinator
    
    Controller
    
    Worker
- Streaming
  Streaming
  - Web-sockets
    Web-sockets
    
    Overview
    
    Quickstart
    
    Client protocol
- kdb Insights Python API
  kdb Insights Python API
- Machine Learning
  Machine Learning
  - About
  - q Interface
    q Interface
    
    About
    
    Analytics
    Analytics
    
    About
    
    ML Analytics API
    ML Analytics API
    
    Introduction
    
    ML Toolkit
    
    Online Models
    Online Models
    
    Introduction
    
    Stochastic Gradient Descent
    Stochastic Gradient Descent
    
    Stochastic Gradient Descent
    
    Linear Regression
    
    Logistic Classification
    
    Secure Updates
    
    Sequential K Means
    
    Variadic Functionality
    Variadic Functionality
    
    Introduction
    
    Function Calls
    
    Clustering models
    
    Statistical models
    
    Time series models
    
    Online models
    
    Registry
    Registry
    
    About
    
    Cloud Integration
    
    Registry API
    Registry API
    
    Storing
    
    Loading
    
    Deleting
    
    Examples
    Examples
    
    Basic Examples
  - Python Interface
    Python Interface
    
    About
    
    Registry
    Registry
    
    About
    
    Cloud Integration
    
    Registry API
    Registry API
    
    Storing
    
    Loading
    
    Deleting
    
    Examples
    Examples
    
    Basic Python Examples
Licensing
Licensing
Product Support
Product Support
- Product Support
- Product Lifecycle