Deploy the ACTIV Market Data Feed Handler with kdb Insights SDK¶
This page details how to run the KX ACTIV Market Data Feed Handler with the kdb Insights SDK in Docker Compose and query the data it stores.
The ACTIV Feed Handler streams real-time market data from the ACTIV One Platform into the kdb Insights SDK, where you can store and query it.
ACTIV deployment process with kdb Insights SDK overview¶
To configure and deploy the feed handler with kdb Insights SDK and query the data it publishes, complete the following tasks:
- Prerequisites
- Log in to the Docker registry
- Download the ACTIV kdb Insights SDK example
- Configure the ACTIV kdb Insights SDK example
- File permissions
- Deploy the ACTIV kdb Insights SDK example
- Query the data
- View the logs
- Tear down the ACTIV kdb Insights SDK example
Architecture¶
The deployment runs the ACTIV feed handler together with the kdb Insights SDK services in Docker Compose. Data flows through the following stages:
- The feed handler publishes ACTIV updates to the
northstream in kdb Insights Reliable Transport (RT), the messaging layer that carries data between components. - The stream processor reads from
north, adds an ingestion timestamp, applies the table schema, and writes to thesouthstream. - The database (storage manager and data access) reads from
southand stores the data. - The gateway serves queries over REST and q interprocess communication (IPC).
RT carries data between these stages. north is the stream the feed handler writes to, and south is the stream the stream processor writes to after it has processed the data.
The following diagram shows how the feed handler publishes to the kdb Insights SDK:

Prerequisites¶
Before running the ACTIV Feed Handler with the kdb Insights SDK, make sure you have the following:
- Docker is installed.
- Docker Compose is installed.
- You have login credentials for the ACTIV feed (host, user ID, and password).
- You have login credentials for the Docker registry at portal.dl.kx.com.
- You have a kdb Insights license file.
Log in to the Docker registry¶
To access the images needed for this walkthrough, log in to the KX Docker registry:
docker login portal.dl.kx.com -u <insert registry username> -p <insert registry password>
Download the ACTIV kdb Insights SDK example¶
To download the ACTIV kdb Insights SDK example, run:
export BEARER=<your bearer token>;\
version=1.15.3;\
curl -L --oauth2-bearer "$BEARER" https://portal.dl.kx.com/assets/raw/kxi-activ-fh/${version}/activ-insights-sdk-example-${version}.tar.gz -o activ-insights-sdk-example.tgz
Portal login
Downloads from the KX portal need a login. Sign in at portal.dl.kx.com, select your email address in the top right to open Token Management, and select Add New Bearer Token. Set BEARER to that token before you run the command. Use the token for downloads from your own machine only, not in a deployment.
Extract (untar) the files and go to the resulting directory:
tar -xvzf activ-insights-sdk-example.tgz;\
cd activ-insights-sdk-example
The directory contains:
| Path | Purpose |
|---|---|
docker-compose-one-node.yaml |
The Docker Compose file for the feed handler and the kdb Insights SDK services. |
env |
The settings and image versions for all containers. |
cfg/ |
The feed handler configuration: kxfeed_config.json, rt_config.json, schema.xml, eventToTable.json, and symbols.txt. |
sp/spec.q, sp/schema.q |
The stream processor pipeline and the table schemas. |
db/config/assembly.yaml |
The database assembly: tables, storage tiers, and retention. |
lic/ |
Place your kdb Insights license file here. |
Configure the ACTIV kdb Insights SDK example¶
Within the activ-insights-sdk-example directory, copy your kdb Insights license file into lic/. The file is named kc.lic or k4.lic, depending on your license type:
cp <path to your license file> lic/
Locate the docker-compose-one-node.yaml file and the env file. Update the following in the env file:
- Update
KXFEED_ACTIV_AOP_HOST,KXFEED_ACTIV_AOP_USERID, andKXFEED_ACTIV_AOP_PASSWORDwith your ACTIV host, user ID, and password. - Check that
kx_licence_dirpoints to the directory containing your license file. The default is./lic.
Add the symbols you want to subscribe to in cfg/symbols.txt, separated by spaces or newlines. To use wildcards such as symbol=GOOGL*, turn on query mode as described in Subscriptions.
File permissions¶
To avoid permission issues with containers mounting directories, run the following within the activ-insights-sdk-example directory. This command might require sudo access:
chmod -R 777 */
Or, use sudo if necessary:
sudo chmod -R 777 */
Deploy the ACTIV kdb Insights SDK example¶
After you update the configuration, load the settings and deploy the ACTIV kdb Insights SDK example:
source ./env
docker compose -f docker-compose-one-node.yaml up
Warning
Use source ./env, not source env. In most shells env also names a system command, which can be loaded instead of your file.
Query the data¶
After deploying, open a second terminal to query the quote data published by the ACTIV feed using this getData REST call:
START=$(date "+%Y.%m.%dD00:00:00.000000000");\
END=$(date "+%Y.%m.%dD23:59:59.999999999");\
curl -X POST "localhost:8080/kxi/getData" -H "Content-Type: application/json" -H "Accept: application/json" --data "{\"table\":\"quote\",\"startTS\":\"$START\",\"endTS\":\"$END\"}"
To query the trade data, use:
START=$(date "+%Y.%m.%dD00:00:00.000000000");\
END=$(date "+%Y.%m.%dD23:59:59.999999999");\
curl -X POST "localhost:8080/kxi/getData" -H "Content-Type: application/json" -H "Accept: application/json" --data "{\"table\":\"trade\",\"startTS\":\"$START\",\"endTS\":\"$END\"}"
Use cancor as the table name to query trade cancellations and corrections. This table fills only when ACTIV sends a trade cancellation or correction, so it's often empty.
Each call returns JSON with a header and a payload. The payload is a list of rows, for example:
{"sym":"SHEL.L","eventType":"EVENT_TYPE_TRADE_NON_REGULAR","updateType":"none","tradeDate":"2026-10-09","tradeTime":"0D09:18:40.948000000","price":3719.5,"updateId":46406,"updateTS":"2026-10-16T08:18:47.263145264"}
"payload":[] with "rc":0 means the query succeeded and no rows matched. Time columns that ACTIV doesn't send are null or empty.
If none of the calls returns any data, review the logs to check for any issues.
Count the rows¶
To see how much data has arrived, count the rows each table returns. This example uses jq:
START=$(date "+%Y.%m.%dD00:00:00.000000000");\
END=$(date "+%Y.%m.%dD23:59:59.999999999");\
for T in quote trade cancor; do
N=$(curl -s -X POST "localhost:8080/kxi/getData" -H "Content-Type: application/json" -H "Accept: application/json" --data "{\"table\":\"$T\",\"startTS\":\"$START\",\"endTS\":\"$END\"}" | jq '.payload | length')
echo "$T rows: $N"
done
Run it twice a few minutes apart. While the market is trading, the quote and trade counts increase.
View the logs¶
To check whether the services started successfully, view the logs using the following command within the activ-insights-sdk-example directory:
docker compose -f docker-compose-one-node.yaml logs | less
To follow a single stage, name its service, for example docker compose -f docker-compose-one-node.yaml logs feed. The services are feed (the feed handler), controller (stream processor controller), worker (stream processor worker), kxi-sm (storage manager), kxi-da (data access), kxi-gw (gateway), kxi-rc (resource coordinator), and kxi-agg (aggregator).
If the ACTIV feed is publishing successfully, the stream processor worker logs show messages such as Received quote or Received trade.
If these messages are absent, search for ERROR and FATAL messages to identify any issues. If no errors are found, validate that it's market hours for the symbols listed in cfg/symbols.txt. A subscription returns an initial snapshot at any time, but trades and quotes arrive only while the market is trading.
At info level, the feed handler log shows Subscribed to <symbol> for each symbol, or Query subscribed to <query> for each query if enableQueries is true. It also logs Assigning symbol <symbol> to <thread> for each subscription, and a summary line, Distributed <n> symbols across <m> worker threads. At debug level it also logs REFRESH received for <symbol> for the initial snapshot. The env file in the example sets the level to debug. To see the individual updates, which the feed handler logs as UPDATE received for <symbol>, set KXFEED_ACTIV_LOGGING_LEVEL=trace in the env file. The output is large, so use it only for short checks.
Replay accounts
If you connect to an ACTIV replay environment, the data is a recorded earlier session that's replayed at the same time of day as the original. Dates in the data, such as tradeDate, are from the recorded day, and you receive updates only during the market hours of the recorded session.
Note
Logs also appear when you run docker compose up, but they can move quickly and be difficult to read. Use docker compose logs for easier reading and searching.
Tear down the ACTIV kdb Insights SDK example¶
To tear down and delete the containers for the ACTIV kdb Insights SDK example, run the following command within the activ-insights-sdk-example directory:
docker compose -f docker-compose-one-node.yaml down
To remove all data written to disk by the example, first delete all volumes:
docker volume rm activ-insights-sdk-example_rt-north-0;\
docker volume rm activ-insights-sdk-example_rt-north-session;\
docker volume rm activ-insights-sdk-example_rt-south-0;\
docker volume rm activ-insights-sdk-example_rt-south-session;
Then, remove all data written to mounted paths (requires sudo access):
sudo bash -c 'rm -rf db/data/db/*; rm -rf db/data/logs/*; rm -rf log/*; rm -rf sp/checkpoints/*'
Customize the example¶
Stream processor pipeline¶
The stream processor runs sp/spec.q. For each of quote, trade, and cancor, it:
- Reads from the
northRT stream and keeps the messages for that table. - Converts the data to a table. The feed handler publishes each batch as a list of columns in the same order as the table in
cfg/schema.xml, and the pipeline names them using the table insp/schema.q. - Adds an
updateTScolumn with the ingestion time. The database uses it as the partition column. - Applies the table schema and writes to the
southRT stream.
Add or change a column¶
To add a column, make the same change in all three places, in the same position:
cfg/schema.xml: add the column with its ACTIV field ID.sp/schema.q: add the column with the matching q type, beforeupdateTS.db/config/assembly.yaml: add the column with the matching type, beforeupdateTS.
The order and types must match, or the stream processor reports a type error and the data doesn't reach the database.