Skip to content

Running RT outside of a container

This page demonstrates how to run the RT service outside of a container. It also includes examples to publish and subscribe to an RT stream using q.

Prerequisites

  • A kdb+ license. If you do not have a license, obtain a trial license on the KX download site.

Download

Step 1 - Obtain a bearer token

  1. Log into the KX downloads portal.
  2. After authentication, navigate to the portal authentication page.
  3. Click Add New Bearer Token.
  4. Click Copy Bearer.

Step 2 - Download the artefacts

Download the kxi-rt.qpk:

version=1.19.0
USERID=<userid for the kx downloads portal>
BEARER=<bearer token>
curl -L  --user $USERID:$BEARER https://portal.dl.kx.com/assets/raw/kxi-rt/$version/kxi-rt.$version.qpk -o kxi-rt.qpk

Start

Step 1 - Install kdb+

You need a kdb+ license to run RT. Follow the install guide to obtain your license and set up your environment.

QHOME=~/q/

Your $QHOME directory should resemble the following structure:

├── kc.lic
├── l64
│   └── q
└── q.k

Step 2 - Start the RT service

A shell script is provided to start your RT stream:

RT Port Usage

RT uses multiple ports. Before starting RT, identify an available port range. RT defines a base port, and the required ports are incremented from this base. The default range includes ports 6000 to 6021.

unzip kxi-rt.qpk
cd kxi-rt

Tip

This script is for illustration purposes and may change over time.

"Usage: start_rt.sh [-p port] [-b base_dir] [-e endpoints] [-t time] [-d disk] [-l limit]"

  1. Start a 3-node RT cluster

    Each RT instance in the cluster is launched with distinct base ports and directories. Each instance is made aware of the other 2 members via the -e endpoints argument.

    ./start_rt.sh -p 6000  -b /tmp/a/ -e "localhost:6001;localhost:16001;localhost:26001"
./start_rt.sh -p 16000 -b /tmp/b/ -e "localhost:6001;localhost:16001;localhost:26001"
./start_rt.sh -p 26000 -b /tmp/c/ -e "localhost:6001;localhost:16001;localhost:26001"

  2. Start a single-node RT cluster

    ./start_rt.sh -p 6000 -b /tmp/d/ -e "localhost:6001"

start_rt.sh accepts the following arguments:

argument default description
p 6000 Base port where RT starts. Ports used increment from this base.
b /tmp/s/ Directory under which RT stores logs. This includes input and output logs. Logs move from the input directory to the output directory after RAFT consensus.
e none RT needs to be told the list of endpoints in the cluster so that connectivity between nodes in the cluster can be established.
t 1440 Retention period for merged log files in minutes. Refer to the RT archival documentation for more information.
d 90 Maximum percentage of disk space used by RT. Refer to the RT archival documentation for more information.
l 10Gi Maximum size of all log files in RT. Refer to the RT archival documentation for more information.

Ports

The list of ports you may need to interact with:

service default description
RT parent process 6001 The parent process responsible for launching the RT sub processes
REST 6002 Any REST requests should be made against this
publisher endpoint 6016 Endpoint to connect to when publishing
subscriber endpoint 6017 Endpoint to connect to when subscribing
Sample REST request example In the 3-node case used above, the REST request could be made against ports 6002, 16002, 26002. 
curl localhost:6002/rt-clients | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   470  100   470    0     0  54900      0 --:--:-- --:--:-- --:--:-- 58750
{
  "result": {
    "node": "LPTP1234",
    "publishers": [
      {
        "pub1.LPTP1234.data": [
          {
            "active": true,
            "connect_time": "2026-03-24T14:42:56.352z",
            "last_message": "2026-03-24T16:59:07.549z",
            "replicated_pos": 3885,
            "total_bytes": 3919,
            "transfer_rate": 0,
            "target_directory": "/tmp/a/in/pub1.LPTP1935.data"
          }
        ]
      }
    ],
    "subscribers": [
      {
        "LPTP1234": [
          {
            "active": true,
            "connect_time": "2026-03-24T14:42:55.719z",
            "last_message": "2026-03-24T16:59:07.552z",
            "replicated_pos": 3870,
            "total_bytes": 3450,
            "transfer_rate": 0
          }
        ]
      }
    ]
  }
}

RT clients

You can use any of the RT interfaces to publish data to your RT stream.

To use the q API for publishing data, you must download the rt.qpk:

version=1.19.0
USERID=<userid for the kx downloads portal>
BEARER=<bearer token>
curl -L --user $USERID:$BEARER  https://portal.dl.kx.com/assets/raw/rt/$version/rt.$version.qpk -o rt.qpk

RT stream connection

The following publisher and subscriber clients connect to RT using the cluster key. This is the endpoint where the RT stream replicators run. There are two server side replicators, push_server for publishers and sub_server for subscribers. With a default RT base port of 6000, these replicators run on ports 6016 and 6017.

Publish

You can use rt.qpk to publish to the RT stream. Start a q session to publish the data:

$ unzip rt.qpk
$ cd rt
$ q startq.q
q)
q)params:(`path`stream`publisher_id`cluster)!("/tmp/rt_pub";"data";"mypub";(":localhost:6016";":localhost:16016";":localhost:26016"));
q)// to connect to a single node RT set the params as follows
q)// params:(`path`stream`cluster`publisher_id)!("/tmp/rt_pub";"data";enlist":localhost:6016";"mypub");
q)p:.rt.pub params;  // initialize your connection to RT. This creates a foreign object, that can be used to publish your data.
q)tab:([] col1:`a`b`c; col2:1 2 3)
q)p(`upd; `tab; tab)  // publish to RT

Data format

RT is data format agnostic. However, to enable future table filtering, publish a three item list. The list must contain the message type, table name, and payload.

Subscribe

You can also use the rt.qpk to subscribe to the RT stream:

$ q startq.q
q)
q)tab:([] col1:`$(); col2:`long$());
q)position:0;
q)upd:{[msg;pos] show `msg`pos!(msg;pos); if[(t:msg 1)in tables[]; upsert[t; msg 2]];}
q)params:`stream`position`callback`cluster!("data"; position; upd; (":localhost:6017";":localhost:16017";":localhost:26017"));
q)s:.rt.sub params;

Payload from RT

When a client subscribes to an RT stream, they define a callback function. This function is triggered whenever RT sends a new message to the subscriber. It accepts two parameters, the first parameter is the message from the publisher, and the second is the position in the RT stream. This position allows subscribers to cache it, and if they restart, resubscribe from the cached position to ensure no data is missed from the RT.

Stop the RT stream

To stop the RT stream, exit the q session.

If you want to clean up the logs, remove the logs underneath the base directory:

rm -r /tmp/s/
Back to top