RT Configuration within kdb Insights Enterprise
Stream log files
When diagnosing issues with a data pipeline that uses an RT cluster (known in the UI as a KX Insights Stream) it is helpful to know where the stream logs reside. This is intended to assist you when the data does not flow to database, allowing you to find the files and identify how far the messages have travelled through the pipeline.
External publishers
component |
file location |
C SDK |
/tmp/rt |
Java SDK |
KXI_LOG_PATH environment variable |
kodbc driver |
/tmp/rt |
The SDKs create a directory name based one the publisher session name and by default the session name is defined as KC_CLIENTID.STREAMID.HOSTNAME. The user can overwrite this session name if necessary to handle deduplication, or have two publishers on the same host.
KX Insights Stream
component |
file location |
Inbound Stream files |
/s/in/KC_CLIENTID.STREAMID.HOSTNAME |
Outbound Stream files |
/s/out/OUT |
Internal Publishers
component |
file location |
Stream Processor |
/sp/checkpoints/rt_pub/{workername}.{ASSEMBLY_NAME}-{streamname} |
Internal Subscribers
component |
file location |
Stream Processor |
/sp/checkpoints/rt_pub/{ASSEMBLY_NAME}-{streamname} |
Storage Manager |
/logs/rt/{ASSEMBLY_NAME}-dap-rdb-[012]/{ASSEMBLY_NAME}-{streamname} |
Cluster RT Ports
port |
type |
notes |
4000 |
TCP |
Sequencer used for the prometheus sidecar |
5000 |
TCP |
Load balancer push server replicator (exposed outside of Kubernetes) |
5001 |
TCP |
Internal pull server |
5002 |
TCP |
Internal push server |
7000 |
UDP |
Raft (exposed to other RT nodes) |
7100 |
TCP |
Raft (exposed to other RT nodes) |
8000 |
UDP |
Sequencer (exposed to other RT nodes) |
9000 |
UDP |
Watcher (exposed to other RT nodes) |