Configuration

A kdb Insights Database's configuration is composed of a schema, a stream, and multiple data tiers for real-time and historical data. In kdb Insights Enterprise, a database can be configured using the Web interface, or using package via the CLI.

At a minimum, an kdb Insights Database needs:

• A gateway for routing and an aggregator to aggregate responses.
• A set of labels to organize and route data queries.
• A schema configuration with one or more tables.
• A source stream to ingest data into the database.
• One or more data tiers for querying data.

Once the database has been configured, the database can be deployed.

Configuration methods

In kdb Insights Enterprise, configuration for a database can be submitted through a package. Databases can also be configured using the Web interface.

Example

Web InterfacePackage shard YAML

Databases are configured through the following tabs on the Databases screen: - Database Settings - Schema Settings - Stream Settings - Resources

Refer to creating a database for full details.

databases/XXX-db/shards/XXX-db-shard.yaml:

name: XXX-db-shard
labels:
  region: us-east-1
mounts:
  rdb:
    type: stream
    baseURI: none
    partition: none
    dependency:
    - idb
  idb:
    type: local
    partition: ordinal
  hdb:
    type: local
    partition: date
    dependency:
    - idb
sm:
  size: 1
  source: south
  tiers:
    - name: streaming
      mount: rdb
    - name: interval
      mount: idb
      schedule:
        freq: 00:10:00
    - name: historical
      mount: hdb
      schedule:
        freq: 1D00:00:00
        snap: 01:35:00
      retain:
        time: 3 Months
daps:
  instances:
    da:
      source: south
      mountList:
      - idb
      - rdb
      - hdb
sequencers:
  south:
    external: false

deployment_config/deployment_config.yaml:

attach: false

tables/trace.yaml:

name: trace
description: Manufacturing trace data
type: partitioned
blockSize: 10000
prtnCol: updateTS
sortColsOrd: [sensorID]
sortColsDisk: [sensorID]
columns:
  - name: sensorID
    description: Sensor Identifier
    type: int
    attrMem: grouped
    attrDisk: parted
    attrOrd: parted
  - name: readTS
    description: Reading timestamp
    type: timestamp
  - name: captureTS
    description: Capture timestamp
    type: timestamp
  - name: valFloat
    description: Sensor value
    type: float
  - name: qual
    description: Reading quality
    type: byte
  - name: alarm
    description: Enumerated alarm flag
    type: byte
  - name: updateTS
    description: Ingestion timestamp
    type: timestamp