Skip to content

Get Started with the DB Service in KDB-X

This page provides a quick introduction to deploying and using the KDB-X DB Service. Both single-node (Docker Compose) and clustered (Kubernetes) options are covered.

Prerequisites

New Community Edition license

If you're using a Community Edition license from before May 2026, you'll need to generate a new one to use the DB Service. You can get a new automatically-generated license by visiting the KDB-X Install page.

Before starting the DB Service, ensure you have the following:

For single-node only:

For clustered only:

Start the DB Service

The single-node DB Service runs as a containerized service and is distributed using Docker Compose.

  1. Clone the DB-Service Github repo.

  2. Log in to the KX docker registry:

    EMAIL=email@example.com
BEARER=BEARERTOKEN
docker login -u $EMAIL -p $BEARER portal.dl.kx.com

  3. Add your license to the .env file:

    export KDB_LICENSE_B64=LICENSE

  4. Run the initialization script - this command initializes the database directories, and copies sample data to the import path:

    ./init-db.sh

  5. Start the service:

    docker compose up -d

Once running, the service is available on port 8080 by default.

The clustered DB Service is deployed to Kubernetes with Helm, as two umbrella charts: one shared gateway, and one release per database shard. The steps below follow the sharded databases reference architecture, which also supplies the assembly and values files.

  1. Clone the DB Service repo and change to the reference architecture directory. Run the remaining commands from there.

    cd referenceArchitectures/helm/sharded-databases

  2. Set your registry credentials, namespace, and license file:

    KX_USER=email@example.com
KX_PASS=BEARERTOKEN
KX_REGISTRY="portal.dl.kx.com"
NAMESPACE="db-service"
LIC_FILE="./k4.lic"

  3. Create the namespace, the image pull secret, and the license secret:

    kubectl create namespace "$NAMESPACE" --dry-run=client -o yaml | kubectl apply -f -

kubectl create secret docker-registry kx-pull-secret \
  --docker-username="$KX_USER" \
  --docker-password="$KX_PASS" \
  --docker-server="$KX_REGISTRY" \
  -n "$NAMESPACE"

kubectl create secret generic kx-license \
  --from-file=license="$LIC_FILE" \
  -n "$NAMESPACE"

  4. Copy the assembly files into the local db chart, then build the chart dependencies:

    cp ./config/*-assembly.yaml ../../kxCharts/db/

helm dependency build ./db-service-api
helm dependency build ./db-service-db-shard

  5. Review the gateway values file before deploying. Its gw.dbsSg.smAddrs entries map each assembly name to the Storage Manager address of its shard, so they must match the Storage Manager service names of the shard releases you are about to create.

  6. Deploy the shared gateway:

    helm install db-service-api ./db-service-api \
  -f ./config/db-service-api-values.yaml \
  -n "$NAMESPACE"

  7. Deploy each database shard, combining the shared values file with that shard's own values file. Set RELEASE_NAME to the shard release you are working with — later commands on this page, and the clustered examples in import, refer to it:

    RELEASE_NAME="<shard-release-name>"

helm install "$RELEASE_NAME" ./db-service-db-shard \
  -f ./config/db-service-db-shard-values.yaml \
  -f ./config/<shard-values-file>.yaml \
  -n "$NAMESPACE"

    Repeat for each shard, changing RELEASE_NAME and the shard values file.

  8. Verify the deployment:

    kubectl get pods -n "$NAMESPACE"

To upgrade later, run helm upgrade with the same values files used at install.

Once all pods are running, the gateway endpoint is determined by your deployment configuration.

To reach the gateway from your local shell without exposing it externally, port-forward its service:

kubectl port-forward svc/db-service-api-gw-sg 8080:8080 -n "$NAMESPACE" &

For full clustered deployment guidance, including a worked multi-shard example, refer to the sharded databases reference architecture.

Once the DB Service is running, you can use one of the connection methods below and try importing and querying some data.

Connect to the DB Service

You can connect to the DB Service using one of the following interfaces:

  • The REST API, using direct HTTP requests.
  • A q client, which wraps the REST API for use in q.
  • A Python client, which wraps the REST API for use in Python.

Select the tab for your preferred client.

dbservice_client.q available via the q client github repo. Install it to your KDB-X module path, by default $HOME/.kx/mod/kx:

cp dbservice_client.q ~/.kx/mod/kx/

Start q and load the client:

/ Load module
dbs:use`kx.dbservice_client

/ Create a session (default endpoint: http://localhost:8080)
session:dbs.createSession[]

/ Or specify an endpoint explicitly (e.g. for clustered)
session:dbs.createSession["your-gateway-host:8080"]

For usage examples and further details, refer to the q client documentation.

  1. Install the Python client:

    pip install --upgrade --pre --extra-index-url https://portal.dl.kx.com/assets/pypi/ kdbx_db_service_client

    For KDB-X Python installation and environment setup, refer to the KDB-X Python install guide.

  2. Create a session:

    import dbservice_client as dbs

# Default endpoint (single-node, localhost)
session = dbs.Session()

# Or specify an endpoint (e.g. for clustered)
session = dbs.Session(host="your-gateway-host", port=8080)

    For non-default endpoints, transport-specific behavior, and examples, refer to the Python client documentation.

Send HTTP requests to the service gateway. For single-node, this is http://localhost:8080 by default. For clustered, use the ingress hostname or load balancer address.

curl -X GET "http://localhost:8080/api/v0/tables"

For full request and response examples, refer to the OpenAPI documentation.

Import sample data

The init-db.sh script (single-node) copies sample data files to the imports staging directory. If you'd like to use your own data, copy it to the data/imports/ folder to make it available to the DB Service. All import paths are relative to the service.

For clustered deployments, copy the file into the /imports directory on the Storage Manager pod of the shard you are importing into. Refer to file staging.

Run the import using one of the supported clients. The examples below import the file fxquote.csv.gz into the fxquote table. The createTable flag creates the table from the incoming data, so the table does not have to exist beforehand.

Clustered deployments

createTable does not work in a clustered deployment. Define fxquote in the assembly YAML before importing into it, and omit createTable from the examples below.

Request

session.importFiles[`table`path`createTable!("fxquote";"fxquote.csv.gz";1b)]

Response

name    | "1d2c6754-1bf8-33a8-8df6-87dc34de40c2"
pipeline| ""
database| "db"
updtype | "ingest"
status  | "pending"
details | ()
tbls    | ()
dates   | ()
progress| `cmdCurrent`cmdIndex`cmdTotal`subCurrent`subIndex`subTotal!("";0n;0n;"";0n;0n)
error   | ""
warnings| ()
updated | "2026-04-27T11:36:26.012305327"

Request

session.import_files(table="fxquote", path="fxquote.csv.gz", createTable=True)

Response

{
  'name': 'b341fb2b-f8ba-4a7e-8b4f-2985ca100ef5',
  'pipeline': '',
  'database': 'db',
  'updtype': 'ingest',
  'status': 'pending',
  'details': [],
  'tbls': [],
  'dates': [],
  'progress': {'cmdCurrent': '',
    'cmdIndex': None,
    'cmdTotal': None,
    'subCurrent': '',
    'subIndex': None,
    'subTotal': None},
  'error': '',
  'warnings': [],
  'updated': '2026-04-27T11:29:38.101833268'
}

Request

curl -s -X POST "http://localhost:8080/api/v0/imports/files" \
  -H "Accept: application/json" \
  -H "Content-Type: application/json" \
  -d '{"table": "fxquote", 
       "path": "fxquote.csv.gz", 
       "createTable": true}'

Response

{
  "name": "27454312-3653-8e62-ded7-8b563a409be6",
  "pipeline": "",
  "database": "db",
  "updtype": "ingest",
  "status": "pending",
  "details": [],
  "tbls": [],
  "dates": [],
  "progress": {
    "cmdCurrent": "",
    "cmdIndex": null,
    "cmdTotal": null,
    "subCurrent": "",
    "subIndex": null,
    "subTotal": null
  },
  "error": "",
  "warnings": [],
  "updated": "2026-04-27T09:48:53.806943488"
}

Query the data

After the import completes, run a query to retrieve data from the fxquote table.

The following examples retrieve rows between 2026-03-02 and 2026-03-03.

Run one of the following examples using your preferred client.

Request

session.querySimple[`table`startTS`endTS`sortCols`limit!(`fxquote;2026.03.02D;2026.03.03D;enlist "ts"; 5)]

Response

trddate    ts                            sym    bid     ask
---------------------------------------------------------------
2026.03.02 2026.03.02D00:00:00.000000000 AUDUSD 0.67091 0.67094
2026.03.02 2026.03.02D00:00:00.000000000 EURUSD 1.16397 1.16399
2026.03.02 2026.03.02D00:00:00.000000000 GBPUSD 1.3419  1.34194
2026.03.02 2026.03.02D00:00:00.000000000 USDCAD 1.38744 1.3875
2026.03.02 2026.03.02D00:00:00.000000000 USDJPY 158.162 158.167

Request

session.query_simple(
    table="fxquote",
    startTS="2026.03.02D00:00:00.000",
    endTS="2026.03.03D00:00:00.000",
    sortCols=["ts"],
    limit=5,
    return_as="json",
)

Response

[
  {'trddate': '2026-03-02',
    'ts': '2026-03-02T00:00:00.000000000',
    'sym': 'AUDUSD',
    'bid': 0.67091,
    'ask': 0.67094},
  {'trddate': '2026-03-02',
    'ts': '2026-03-02T00:00:00.000000000',
    'sym': 'EURUSD',
    'bid': 1.16397,
    'ask': 1.16399},
  {'trddate': '2026-03-02',
    'ts': '2026-03-02T00:00:00.000000000',
    'sym': 'GBPUSD',
    'bid': 1.3419,
    'ask': 1.34194},
  {'trddate': '2026-03-02',
    'ts': '2026-03-02T00:00:00.000000000',
    'sym': 'USDCAD',
    'bid': 1.38744,
    'ask': 1.3875},
  {'trddate': '2026-03-02',
    'ts': '2026-03-02T00:00:00.000000000',
    'sym': 'USDJPY',
    'bid': 158.162,
    'ask': 158.167}
]

Request

curl -s -X POST "http://localhost:8080/api/v0/query/simple" \
  -H "Accept: application/json" \
  -H "Content-Type: application/json" \
  -d '{"table": "fxquote", 
      "startTS": "2026.03.02D00:00:00", 
      "endTS": "2026.03.03D00:00:00", 
      "limit": 2}'

Response

{
  "header": {
    "corr": "43bb92c3-2d55-488c-8a38-adbf04e3f506",
    "logCorr": "43bb92c3-2d55-488c-8a38-adbf04e3f506",
    "version": 1,
    "rcvTS": "2026-04-27T09:53:17.384000000",
    "http": "json",
    "api": ".query.simple",
    "agg": ":172.20.0.7:5060",
    "refVintage": -9223372036854775807,
    "rc": 0,
    "ac": 0,
    "ai": "",
    "limitApplied": true
  },
  "payload": [
    {
      "trddate": "2026-03-02",
      "ts": "2026-03-02T00:00:00.000000000",
      "sym": "AUDUSD",
      "bid": 0.67091,
      "ask": 0.67094
    },
    {
      "trddate": "2026-03-02",
      "ts": "2026-03-02T00:00:06.000000000",
      "sym": "AUDUSD",
      "bid": 0.67095,
      "ask": 0.67097
    }
  ]
}

Reset the service (single-node only)

Important

Resetting the service permanently deletes all data and RT client logs, and re-initializes directories.

If you need to reset the database, run:

docker compose down 
./reset-db.sh

For clustered deployments, delete the Helm releases instead:

helm delete "$RELEASE_NAME" -n "$NAMESPACE"

Deleting a Helm release does not delete its volumes, so data persists across redeployments. Delete retained volumes manually when they are no longer required:

kubectl delete pvc "${RELEASE_NAME}-imports" -n "$NAMESPACE"

Next steps