Skip to content

Query Data in KDB-X DB Service

This page provides an overview of how to query data in KDB-X DB Service. The query APIs and behavior are the same for both single-node and clustered deployments, with a few differences in how the gateway routes and aggregates results.

KDB-X DB Service provides 4 query interfaces for retrieving data.

  • Structured query — parameterized API for querying timeseries.

  • q query — free-form q/qsql, which can include an optional free-form aggregation function.

  • SQL querySELECT-only SQL interface.

  • Preview query — lightweight sample of a table, for reviewing a small subset of the data or checking schema compatibility.

A detailed reference, including examples, for all APIs is available in the API spec. Sample notebooks demonstrating API usage are bundled with the DB Service repo on GitHub.

Choosing a query method

The query interfaces cover different trade-offs. Use this table to pick the right one for your use case.

Interface Best for Notes
Structured query Most use cases Parameterized API — no query language required. Handles tier routing, aggregation, filtering, and sorting through explicit parameters.
SQL Familiarity, ad-hoc exploration Standard SELECT syntax. Runs across all tiers automatically. Good choice if your team prefers SQL over q.
q query Complex custom logic Full q expressiveness. Required when structured or SQL can't express what you need. Use .kxi.selectTable to safely query across all tiers.
Preview query Inspecting data or checking a schema Returns a small sample of a table with minimal time and resource usage. Not for retrieving a specific result set.

REST vs IPC is a separate decision from which query interface to use:

  • REST — works over HTTP from any language. Recommended for most integrations. The Python and q clients both wrap the REST API.
  • IPC (q only) — direct socket connection from a q process. Lower overhead for high-frequency queries from q.

For a detailed comparison of REST vs IPC performance trade-offs, see the Insights query methods overview.

Query routing

In both deployment types, the service gateway handles query routing automatically.

The gateway routes queries across the RDB, IDB, and HDB tiers based on the time range specified. Results are aggregated before being returned to the client.

The gateway routes queries across data tiers (RDB, IDB, HDB) and also across shards. When a query spans multiple shards, each shard executes it independently and the gateway aggregates the results.

A clustered deployment can hold several assemblies of the same logical database, so a query can narrow which of them it targets. Two parameters do this:

Parameter Targets
scope A single assembly, by its exact name: "scope": {"assembly": "marketdata-emea"}
labels Every assembly matching the given labels: "labels": {"assetClass": "fx", "region": "emea"}

To target one specific assembly, either works — use scope if you know its name, or labels if you would rather select it by its metadata. To target more than one at once, use labels and give only the labels they share: {"assetClass": "fx"} matches every FX assembly. Omitting both queries across all assemblies.

{"table": "fx_trade",
 "startTS": "2026.01.21D10:00:00.000000000",
 "endTS": "2026.01.21D10:00:04.000000000",
 "labels": {"assetClass": "fx", "region": "emea"}}

Support varies by API:

API scope labels
Structured query Supported Supported
q query Supported Supported
Preview query Supported Supported
SQL query Supported Not supported

Import requests target an assembly differently again, with the assembly query parameter rather than a request body field.

Query results

All queries return a header and a payload.

  • The header contains execution metadata, including correlation IDs, timestamps, the API used, and whether limits were applied. This information can be used for debugging, tracing requests, and understanding how the query was processed.

  • Errors are reported in the ai field in the header.

  • The payload contains the result rows, returned as JSON. The q and Python clients return them as tables by default, but this can be overridden to JSON (or a Pandas dataframe in the Python client).

{
  "header": {
    "corr": "740fb465-208b-4be1-acc9-3424136cdcd5",
    "logCorr": "740fb465-208b-4be1-acc9-3424136cdcd5",
    "version": 1,
    "rcvTS": "2026-04-27T09:57:18.238000000",
    "http": "json",
    "api": ".query.q",
    "agg": ":172.20.0.3:5060",
    "refVintage": -9223372036854775807,
    "rc": 0,
    "ac": 0,
    "ai": ""
  },
  "payload": [
    {
      "trddate": "2026-01-21",
      "sym": "EURUSD",
      "o": 901.2,
      "h": 901.2,
      "l": 901.2,
      "c": 901.2
    },
    ...
  ]
}

Structured query

The Structured query endpoint is POST /api/v0/query/simple.

Structured query uses explicit parameters instead of a query string.

  • table is required for all queries.
  • startTS and endTS timestamps are required for all timeseries requests; they are used to route requests to the appropriate DB tier (RDB, IDB, HDB) and locate data on disk in the correct partition. These values work on the prtnCol defined for the table.

Filtering

The filter parameter takes a list of filter triples in the form [operator, column, values]. It is possible to nest filters with the logical operators not, and, or. For example, ["and", ["=", "size", 100], ["<", "price", 500]].

Operator Notes
=, <>, <, >, <=, >= Standard comparison
in Expects a list of values
within Expects a range as a tuple
like Expects a symbol or string; follows KDB-X like rules

Filters can be applied to reference columns as well as to columns of the queried table.

Aggregation

The agg parameter has three different behaviors, depending on what type of list it's given:

Input type Behavior Example
List of column names Specifies what columns to include in the result ["sym", "bid", "ask"]
List of tuples Specifies what columns to include in the result, and renames them [["Symbol", "sym"], ["BidPrice", "bid"], ["AskPrice", "ask"]]
List of triples Applies aggregation function to column and renames it [["o", "first", "bid"], ["h", "max", "bid"], ["l", "min", "bid"], ["c", "last", "bid"]]

Available aggregation functions: all, any, avg, cor, count, cov, dev, distinct (always returns a list), first, last, max, min, prd, scov, sdev, sum, svar, var, wavg, wsum.

Use groupBy to group aggregated results by another column.

Reference columns

Where a column of the queried table declares a foreign key, a query can read columns of the referenced table by naming them in table.column form; the service resolves the foreign key and joins the reference data for you. A structured query accepts reference columns in agg, groupBy and filter, and returns them under their dotted name:

{"ts":"2026-03-02T00:00:00.000000000","sym":"EURUSD","bid":1.16397,"ask":1.16399,"instruments.category":"Major"}

Refer to querying reference data for a worked example.

A q query reads reference columns through .kxi.selectTable, naming them the same way in its functional agg, filter and groupBy clauses:

.kxi.selectTable[`table`startTS`endTS`filter`agg!(
    `fxquote;
    2026.03.02D00:00:00;
    2026.03.02D00:00:10;
    enlist(=;`instruments.category;enlist`Major);
    (`ts`sym`cat)!(`ts;`sym;`instruments.category))]

Support for reference columns varies by API:

API Reference columns
Structured query, in agg, groupBy and filter Supported
q query through .kxi.selectTable, in agg, groupBy and filter Supported
q query selecting from the table directly, for example select instruments.category from fxquote Not supported; the query signals an error
SQL query, in dotted form Not supported; use an explicit join

Temporal dot notation is not supported

Because the dot operator is interpreted as a reference to another table, the KDB-X dot notation for temporal components — for example timestamp.second or timestamp.minute — is not supported in a structured query.

Sorting

The sortCols parameter specifies a list of sorting criteria for the result. Sorting is ascending by default.

Input type Behavior Example
List of column name strings Sort ascending by each column ["sym", "ts"]
List of tuples [col, direction] Sort each column in the specified direction [["sym", "asc"], ["ts", "desc"]]
List of strings with "desc" appended Sort all columns descending ["sym", "ts", "desc"]

Other features

Temporality

The temporality parameter controls how the time range is applied.

Value Behavior
snapshot (default) If temporality is not specified, returns a continuous block of data between startTS and endTS
slice Returns data between the time of startTS and time of endTS for every day in the range of startTS to endTS. For example, given a startTS of 2021.01.01D10:00:00 and an endTS of 2021.01.03D13:00:00, data would be returned from 10:00:00 and 13:00:00 on each of the dates 2021.01.01, 2021.01.02, 2021.01.03.

Fill

The fill parameter replaces null values in the result.

Value Behavior
forward Replaces nulls with the previous non-null value
zero Replaces nulls with the zero-equivalent for the column type

Limit

The limit parameter restricts the number of rows returned.

Value Behavior Example
Positive integer Returns rows counting forward from the start of the result 100 returns the first 100 rows
Negative integer Returns rows counting backward from the end of the result -100 returns the last 100 rows
Tuple of integers Skips the number of rows given by the first integer, then returns the number of rows given by the second. Use this to page through a large result. [200, 100] skips the first 200 rows and returns the next 100

Timezone handling

All timestamps in KDB-X DB Service data are treated as UTC. Structured queries support explicit timezone control via three parameters:

Parameter Description
inputTZ Timezone to interpret startTS and endTS as. Defaults to UTC.
outputTZ Timezone to convert outputTZCols to. Defaults to UTC (no conversion).
outputTZCols List of columns to convert to the output timezone outputTZ.

Timezones should be supplied in tz database format; for example America/New_York or Europe/London.

Example structured query

session.querySimple[([
    table:`fxquote;
    startTS:2026.03.02D00:00:00;
    endTS:2026.03.04D00:00:00;
    filter:enlist[("in";"sym";(`EURUSD;`USDJPY;`GBPUSD))];
    groupBy:(`trddate;`sym);
    agg:((`o;`first;`bid);
        (`h;`max;`bid);
        (`l;`min;`bid);
        (`c;`last;`bid))])]
session.query_simple(table='fxquote',
    startTS='2026.03.02D00:00:00',
    endTS='2026.03.04D00:00:00',
    filter=[['in','sym',['EURUSD','USDJPY','GBPUSD']]],
    groupBy=['trddate','sym'],
    agg=[['o', 'first', 'bid'],
        ['h', 'max', 'bid'],
        ['l', 'min', 'bid'],
        ['c', 'last', 'bid']],
    return_as="pandas")
curl -X POST "http://localhost:8080/api/v0/query/simple" \
    -H "Content-Type: application/json" \
    -d '{"table":"fxquote",
        "startTS":"2026.03.02D00:00:00",
        "endTS":"2026.03.04D00:00:00",
        "filter":[["in","sym",["EURUSD","USDJPY","GBPUSD"]]],
        "groupBy":["trddate","sym"],
        "agg":[["o", "first", "bid"],
                ["h", "max", "bid"],
                ["l", "min", "bid"],
                ["c", "last", "bid"]]}'

q query

The q query endpoint is POST /api/v0/query/q. It has a single required parameter, query, containing free-form q. It can be used for more complex queries than the Structured or SQL endpoints.

The query is not routed to one tier. The gateway runs it independently on every DB tierRDB, IDB, and HDB — and then combines the per-tier results into a single response, as described in the next section.

Aggregation

By default, the gateway will concatenate the results from each tier (RDB, IDB, HDB) by performing a raze. The optional parameter agg specifies a q lambda to be used instead, and allows for more sophisticated aggregation and data shaping of the final result.

It can be specified as either:

  • A unary function, for example "distinct".
  • A composition of named unary functions, for example "'[distinct;raze]".
  • A unary lambda, for example "{distinct raze x}".

The data passed to the function is a list of query results, each element being the query results from one tier. For example, "{distinct raze x}" razes the per-tier results into a single table and then removes duplicate rows.

Clustered deployments

In a clustered deployment with sharding, agg is applied once, across the combined results from every tier and every shard — not per shard and then again across shards. A custom agg function therefore receives one list containing all of those results.

.kxi.selectTable

The DB Service stores data across multiple internal tables and uses additional metadata columns to handle late data and query availability during end-of-interval (EOI) and end-of-day (EOD) processing. The helper function .kxi.selectTable accounts for this: it returns all of the data for a table, and hides the internal metadata columns.

Use .kxi.selectTable for q queries unless you are querying historical HDB data only and late data is not a concern. Selecting from a table directly may otherwise cause the following issues:

  • Missing late data. The result may omit records that arrived late.
  • Missed reference data updates. The result may not reflect updates made to reference tables between EOIs.
  • Schema mismatches on aggregation. A wildcard select * returns the internal metadata columns as well as the table's own columns. These metadata columns differ between the RDB, IDB, and HDB, so the per-tier results have different schemas and the default raze cannot combine them.
  • No forward compatibility. The internal tables and metadata columns may change in future releases, so queries that read them directly can break when you upgrade.

Structured and SQL queries are unaffected

Structured and SQL queries are not affected by these issues.

.kxi.selectTable takes a single argument, a dictionary with the following parameters:

Field Type Required Details Default
table symbol Yes Table name.
startTS timestamp Start time (inclusive) for selecting data. This can be an empty list for reference tables. -0Wp
endTS timestamp End time (exclusive) for selecting data. This can be an empty list for reference tables. 0Wp
filter list Functional where clause, for example, a list of the parse tree. (), indicating no filter is applied.
groupBy dict/boolean Functional by clause. 0b, indicating no by clause.
agg dict Functional aggregation/select clause. All columns of the table are included.
mmap boolean Boolean that indicates whether the result of a .kxi.selectTable is mapped instead of loaded to memory. 0b, indicating that this feature is disabled.
limit int int[2] Value that limits the result row number. (Ordering not guaranteed).

The correct forms for filter, groupBy, and agg are different than the structured query. Using parse on a direct select query can be a useful guide.

.kxi.selectTable also resolves reference columns, which a direct select cannot.

Example q query

// Using .kxi.selectTable - safest
session.queryQ["
.kxi.selectTable[([
    table:`fxquote;
    startTS:2026.03.02D00:00:00;
    endTS:2026.03.04D00:00:00;
    filter:((in;`sym;enlist (`EURUSD;`USDJPY;`GBPUSD));(in;`trddate;2026.03.02 2026.03.03));
    groupBy:`trddate`sym!`trddate`sym;
    agg:`o`h`l`c!((*:;`bid);(max;`bid);(min;`bid);(last;`bid))])]"]

// Selecting data directly - potentially unsafe
session.queryQ["
select o: first bid,
        h: max bid,
        l: min bid,
        c: last bid 
by trddate, sym 
from fxquote 
where sym in (`EURUSD;`USDJPY;`GBPUSD),
    trddate in 2026.03.02 2026.03.03"]
# Using .kxi.selectTable - safest
session.query_q(query="""
.kxi.selectTable[([
    table:`fxquote;
    startTS:2026.03.02D00:00:00;
    endTS:2026.03.04D00:00:00;
    filter:((in;`sym;enlist (`EURUSD;`USDJPY;`GBPUSD));(in;`trddate;2026.03.02 2026.03.03));
    groupBy:`trddate`sym!`trddate`sym;
    agg:`o`h`l`c!((*:;`bid);(max;`bid);(min;`bid);(last;`bid))])]
""", return_as="pandas")

# Selecting data directly - potentially unsafe
session.query_q(query="""
select o: first bid,
        h: max bid,
        l: min bid,
        c: last bid 
by trddate, sym 
from fxquote 
where sym in (`EURUSD;`USDJPY;`GBPUSD),
    trddate in 2026.03.02 2026.03.03
""", return_as="pandas")
# Using .kxi.selectTable - safest
QUERY='
.kxi.selectTable[([
    table:`fxquote;
    startTS:2026.03.02D00:00:00;
    endTS:2026.03.04D00:00:00;
    filter:((in;`sym;enlist (`EURUSD;`USDJPY;`GBPUSD));(in;`trddate;2026.03.02 2026.03.03));
    groupBy:`trddate`sym!`trddate`sym;
    agg:`o`h`l`c!((*:;`bid);(max;`bid);(min;`bid);(last;`bid))])]
'
curl -s -X POST "http://localhost:8080/api/v0/query/q" \
    -H "Content-Type: application/json" \
    -d "{\"query\":\"$QUERY\"}"

# Selecting data directly - potentially unsafe
QUERY='
select o: first bid,
        h: max bid,
        l: min bid,
        c: last bid 
by trddate, sym 
from fxquote 
where sym in (`EURUSD;`USDJPY;`GBPUSD),
    trddate in 2026.03.02 2026.03.03
'
curl -s -X POST "http://localhost:8080/api/v0/query/q" \
    -H "Content-Type: application/json" \
    -d "{\"query\":\"$QUERY\"}"

SQL query

The SQL query endpoint is POST /api/v0/query/sql. The KDB-X DB Service runs SQL queries across all data tiers, and supports a subset of the KDB-X SQL dialect. Queries are limited to SELECT, following the following structure:

SELECT [DISTINCT] ... FROM TABLE [WHERE ...] [GROUP BY ...] [HAVING ...] [ORDER BY ...] [LIMIT ...] [OFFSET ...]

Limitations:

  • INSERT, CREATE, and DELETE are not supported by the API.
  • Chained boolean operations are not supported, logic must be wrapped in parenthesis. For example: SELECT sym, (sector='tech') or ((exchangeID in (1,2)) or (sym='FNLH.CAN')) FROM instrument.
  • Positional grouping is not supported, for example SELECT avg(price) FROM trade GROUP BY 1.
  • Non-aggregate operations on the GROUP BY columns are not supported.
  • Irreversable operations are not permitted in WHERE clauses when restricting on the partition column (date). For example: SELECT * FROM myTable WHERE date % 2 = 0.

Joins

The SQL interface supports INNER, LEFT, RIGHT, FULL, and CROSS joins. Note the following constraints:

Constraint Detail
Aliases Required when using joins
Join condition Only = is supported for joining columns
Syntax Must use the ON keyword
Column types Left-hand and right-hand join columns must be the same kdb+ datatype

Set operations

UNION, INTERSECT and EXCEPT operators are supported to combine the results of two queries. Note the following constraints:

Constraint Detail
Compatibility Both SELECT statements must return the same number of columns with the same data types
Result ordering The order of data returned from a set operation query is not guaranteed unless the ORDER BY clause is included in the query.
Supported clauses on result Currently only LIMIT and ORDER BY are supported on the result set of a UNION, EXCEPT, INTERSECT query.

Supported functions

Aggregation

count, first, last, min, max, sum, avg, prod, wsum, wavg, var, svar, dev, stdev

Comparison operators

Operator Notes
= Exact match on strings. Use like for pattern matching.
!=, >, <, >=, <= Standard comparisons. Note: > does not work on alphabetical lists.
!<, !> Not less than / not greater than
in Match against a list of values
between Range comparison (inclusive)
like Pattern matching using KDB-X like rules — does not support full regex
is null, is not null Null checks

Mathematical functions

Function Notes
abs, neg, ceil / ceiling, floor, round, trunc Rounding and sign
trunc(a, b) Truncates a to b decimal places
sqrt, cbrt Square and cube root. Does not support negative numbers.
exp, ln, log, log10, power Exponential and logarithmic
sin, cos, tan, asin, acos, atan, atan2 Trigonometric (radians)
sind, cosd, tand, asind, acosd, atand, atan2d Trigonometric (degrees)
sinh, cosh, tanh, asinh, acosh, atanh Hyperbolic
degrees, radians, pi Angle conversion and constant
div, gcd, lcm, factorial Integer arithmetic
width_bucket(a, b) Returns the bucket that value a belongs to in list b
width_bucket(a, b, c, d) Returns the bucket for a between min b and max c, with d total buckets

Mathematical operators

Operator Meaning
+, -, *, / Arithmetic
% Modulo
^ Exponent
@ Absolute value
\|/ Square root
\|\|/ Cube root
&, \|, ~, # Bitwise AND, OR, NOT, XOR
<<, >> Bitwise left shift, right shift

Value functions

current_date, current_timestamp, current_time

Miscellaneous functions

Function Notes
cast Casting to binary, varbinary, and image is not supported
date_trunc Calling date_trunc("millennium", time) on dates before 2000.01.01 returns -0W (negative infinity), as KDB-X cannot represent year 1001.01.01
date_part, extract Date component extraction
coalesce, nullif, case Conditional / null handling
concat, cat, upper, lower, trim, ltrim, rtrim, btrim String manipulation
left, right, length, char_length, character_length, bit_length, octet_length String length and slicing
position, starts_with, reverse String search
now, random, setseed, version Utility functions

Example SQL query

Calculating OHLC bars using the different methods:

query:"SELECT
    trddate,
    sym,
    FIRST(bid) AS o, 
    MAX(bid) AS h, 
    MIN(bid) AS l, 
    LAST(bid) AS c 
FROM fxquote 
WHERE 
    trddate in ('2026-03-02', '2026-03-03') AND
    sym in ('EURUSD','USDJPY','GBPUSD')
GROUP BY trddate, sym;"
session.querySQL[query]
query = """
SELECT 
    trddate,
    sym,
    FIRST(bid) AS o, 
    MAX(bid) AS h, 
    MIN(bid) AS l, 
    LAST(bid) AS c 
FROM fxquote 
WHERE 
    trddate in ('2026-03-02', '2026-03-03') AND
    sym in ('EURUSD','USDJPY','GBPUSD')
GROUP BY trddate, sym;
"""
session.query_sql(query, return_as="pandas")
QUERY="
SELECT 
    trddate,
    sym,
    FIRST(bid) AS o, 
    MAX(bid) AS h, 
    MIN(bid) AS l, 
    LAST(bid) AS c 
FROM fxquote 
WHERE 
    trddate in ('2026-03-02', '2026-03-03') AND
    sym in ('EURUSD','USDJPY','GBPUSD')
GROUP BY trddate, sym;
"
curl -s -X POST "http://localhost:8080/api/v0/query/sql" \
-H "Content-Type: application/json" \
-d "{\"query\":\"$QUERY\"}"

Preview query

The Preview query endpoint is POST /api/v0/query/preview.

The preview API is a lightweight data retrieval API that retrieves a small sample of table data with minimal time and resource usage. Use it to inspect data or test software compatibility with a schema.

Parameters

Name Required Type Description
table yes symbol Name of table to retrieve data from.
startTS no timestamp Inclusive start time of the period of interest. This field is optional. By default, the preview covers the full temporal range of the database. The timestamp must be midnight so the range covers full days.
endTS no timestamp Exclusive end time of period of interest. This field is optional. By default, covers the full temporal range of the database. The timestamp must be midnight so the range covers full days.
limit no long Maximum number of rows to return. The default limit is 1000.

Result

The preview API returns up to limit rows from the specified table and time range. It searches all available data before returning results. If fewer than limit rows are returned, the response contains all available rows. Otherwise, the API does not guarantee which rows are returned, and the selection may vary depending on the implementation.

Example preview query

session.queryPreview[([
    table:`fxquote;
    startTS:2026.03.02D00:00:00;
    endTS:2026.03.04D00:00:00;
    limit:100])]
session.query_preview(table='fxquote',
    startTS='2026.03.02D00:00:00',
    endTS='2026.03.04D00:00:00',
    limit=100,
    return_as="pandas")
curl -s -X POST "http://localhost:8080/api/v0/query/preview" \
    -H "Content-Type: application/json" \
    -d '{"table":"fxquote",
        "startTS":"2026.03.02D00:00:00",
        "endTS":"2026.03.04D00:00:00",
        "limit":100}'

Next steps