Skip to content

Troubleshooting

This page covers common issues with the KDB-X DB Service and how to diagnose them.

Start with the general tips below to establish which component is at fault, then work through the section that matches your symptom: import, query, deployment, or connection issues. If none of them resolve the problem, see Getting help.

Commands are given for both deployment types where they differ — kubectl for clustered deployments and docker compose for single-node. The kubectl commands assume $NAMESPACE is set to the namespace you deployed into, as in the quickstart.

General tips

These checks apply to most problems and are usually the quickest way to narrow down the cause.

  • Check pod status first (clustered). Most issues surface as pods in a non-running state.

    kubectl get pods --namespace "$NAMESPACE"
kubectl describe pod <pod-name> --namespace "$NAMESPACE"

  • Check service logs. The gateway, storage manager, and DAP logs contain the most useful diagnostic information.

    kubectl logs -l component=gateway --namespace "$NAMESPACE"
kubectl logs -l component=storage-manager --namespace "$NAMESPACE"
kubectl logs -l component=dap --namespace "$NAMESPACE"

  • Check Docker Compose status (single-node).

    docker compose ps
docker compose logs

  • Verify connectivity. Confirm the gateway is reachable before debugging query or ingest issues. A table listing is the lightest request that confirms the gateway is up.

    curl http://localhost:8080/api/v0/tables

Import issues

Job stays in pending or processing state

The import job is running but not completing. Check the storage manager logs for errors:

kubectl logs -l component=storage-manager --namespace "$NAMESPACE" | grep -i error

Common causes: insufficient disk space, file not found at the specified path, or a malformed file.

File not found error

The file path specified in the import request is not accessible to the DB Service.

  • For single-node: ensure the file is in data/imports/ relative to the Docker Compose directory.
  • For clustered: ensure the file has been copied into the /imports directory on the Storage Manager pod of the shard you are importing into, and that the request path is relative to that directory. Refer to file staging.

Schema mismatch on import

The file's column names or types do not match the table definition. Check the table schema:

session.describeTable[`fxquote]

curl "http://localhost:8080/api/v0/tables/fxquote"

If using createTable: true, the table was created from the file's inferred schema. Drop and recreate the table with explicit column definitions if needed.

Sym file errors (initial import)

If running an initial import, the sym file must be at the root of the staged HDB directory. Missing or mismatched sym files cause partition registration failures. See Initial Import for more detail.

Query issues

Query returns no data

  1. Confirm the table exists and has data:

    curl http://localhost:8080/api/v0/tables

  2. Check that startTS and endTS cover the date range where data was imported. Timestamps are always UTC.

  3. If using q query directly (without .kxi.selectTable), data in the IDB may be missed. Use .kxi.selectTable to query across all tiers correctly — see q query.

Timeout errors

Queries exceeding the gateway timeout return a timeout error. For large queries:

  • Add filters to reduce data volume.
  • Retrieve results in pages, using the tuple form of limit to skip a number of rows and then return the next batch — see Limit.
  • For q queries, check that your aggregation function is not blocking on large results.

DAP not available error

No DAP is ready to serve the query. Check DAP pod status:

kubectl get pods -l component=dap --namespace "$NAMESPACE"

DAPs start up after the storage manager and may take a minute to register. Wait for all pods to reach Running and retry.

Unexpected query results or missing columns

If q queries return inconsistent results or missing columns across tiers (RDB vs IDB vs HDB), you may be querying the tiers directly rather than using .kxi.selectTable. See q query for the correct usage.

Deployment issues (clustered)

Pods stuck in Pending state

Usually a resource scheduling issue. Check events:

kubectl describe pod <pod-name> --namespace "$NAMESPACE"

Common causes: insufficient CPU/memory on cluster nodes, missing secrets, or PVC not bound.

ImagePullBackOff error

The cluster cannot pull the DB Service image. Verify the registry secret is correctly configured:

kubectl get secret kx-pull-secret --namespace "$NAMESPACE"

Recreate the secret if expired or incorrect — refer to Quickstart.

Assembly YAML validation errors

Helm rejects the assembly on upgrade. Re-run the same helm upgrade you used to apply the change, with --dry-run added, and check the error output — --dry-run validates the assembly without applying changes. Refer to Applying changes.

Connection issues

Cannot connect to the gateway

  • Single-node: confirm Docker Compose is running (docker compose ps) and the service is on port 8080.

  • Clustered: check that the ingress or load balancer is configured and that the gateway service has an external IP.

    kubectl get svc --namespace "$NAMESPACE"

Authentication errors

Check that your license and registry credentials are correct. For the KDB-X license, verify the secret exists in the namespace:

kubectl get secret kx-license --namespace "$NAMESPACE"

Getting help

If you cannot resolve an issue using this guide: