RT Configuration within kdb Insights Enterprise

Stream log files

When diagnosing issues with a data pipeline that uses an RT cluster (known in the UI as a KX Insights Stream) it is helpful to know where the stream logs reside. This is intended to assist you when the data does not flow to database, allowing you to find the files and identify how far the messages have travelled through the pipeline.

External publishers

component	file location
C SDK	/tmp/rt
Java SDK	KXI_LOG_PATH environment variable
kodbc driver	/tmp/rt

The SDKs create a directory name based one the publisher session name and by default the session name is defined as KC_CLIENTID.STREAMID.HOSTNAME. The user can overwrite this session name if necessary to handle deduplication, or have two publishers on the same host.

KX Insights Stream

component	file location
Inbound Stream files	/s/in/KC_CLIENTID.STREAMID.HOSTNAME
Outbound Stream files	/s/out/OUT

Internal Publishers

component	file location
Stream Processor	/sp/checkpoints/rt_pub/{workername}.{ASSEMBLY_NAME}-{streamname}

Internal Subscribers

component	file location
Stream Processor	/sp/checkpoints/rt_pub/{ASSEMBLY_NAME}-{streamname}
Storage Manager	/logs/rt/{ASSEMBLY_NAME}-dap-rdb-[012]/{ASSEMBLY_NAME}-{streamname}

Cluster RT Ports

port	type	notes
4000	TCP	Sequencer used for the prometheus sidecar
5000	TCP	Load balancer push server replicator (exposed outside of Kubernetes)
5001	TCP	Internal pull server
5002	TCP	Internal push server
7000	UDP	Raft (exposed to other RT nodes)
7100	TCP	Raft (exposed to other RT nodes)
8000	UDP	Sequencer (exposed to other RT nodes)
9000	UDP	Watcher (exposed to other RT nodes)