Reliable Transport (RT) files
RT is a microarchitecture for building reliable streaming applications on top of kdb+. It is the native message transport for KX Insights, and it is designed to satisfy both high availability and high performance requirements within the KX Insights Platform.
RT uses a sequencer and replication strategy to ensure that messages are moved as quickly as possible between publishers and subscribers. Its basic operation consists of the following steps:
- Publishers write events to a transaction stream file on local disk.
- The stream file is replicated between publisher and the RT cluster.
- Subscribers consume the transaction stream file once it has been replicated and processed by RT.
When diagnosing issues with a pipeline that uses RT it is helpful to know where the stream logs reside. This is intended to assist you when the data does not flow to database, allowing you to find the files and identify how far the messages have travelled through the pipeline.
Stream logs
External publishers
| component | file location |
|---|---|
| ODBC driver on Linux | /tmp/rt |
| Java SDK on Linux | KXI_LOG_PATH environment variable |
| ODBC driver or Java SDK on Windows | Path defined in HKEY_CURRENT_USER\SOFTWARE\kx\kodbcDriver\LogPath |
Java SDK
Currently the Java SDK does not truncate the stream logs once they have been replicated inside the Insights Platform. Run this in each directory to truncate them:
find . -perm -1000 -type f -exec truncate --size 0 {} \;RT Stream
| component | file location |
|---|---|
| Inbound RT Stream files | /s/in/$TOPIC |
| Outbound RT Stream files | /s/out/OUT |
Outbound RT Stream files
Currently the RT stream log archival rules do not get applied to the outbound files. Run this on each RT node to delete log files older than 7 days (your choice):
find /s/out/OUT -mtime +7 -exec rm -f {} \;Internal Publishers
| component | file location |
|---|---|
| Stream Processor | /sp/checkpoints/rt_pub/\({workername}.\)-${streamname} |
Warning
Currently the stream logs are not truncated once they have been replicated. Run this in each directory to delete all files older than 7 days (you may change the number of days if you wish):
find . -mtime +7 -exec rm -f {} \;Internal Subscribers
| component | file location |
|---|---|
| Stream Processor | /sp/checkpoints/rt_pub/\({ASSEMBLY_NAME}-\) |
| Storage Manager | /logs/rt/{ASSEMBLY_NAME}-dap-rdb-[012]/{ASSEMBLY_NAME}-${streamname} |
Warning
Currently the RT stream log archival rules do not get applied to the subscriber files. Run this in each directory to delete files older than 7 days (you may change the number of days if you wish):
find . -mtime +7 -exec rm -f {} \;Application logs
RT application logs contain messages reflecting the normal operation along with unexpected conditions.
There are two set of application logs: replicator and sequencer.
Replicator
The replicator application logs are either located in a subdirectory replicator which is in the same directory as the RT stream log files (in the case of push_client, pull_client), or the directory above the RT stream log files (in the case of push_server, pull_server).
The filenames have the format: <replicator_type>.<pid> where replicator type is pull_server/push_server/pull_client/push_client.
An RT node can have multiple replicator processes running at the same time so there may be number of application log files.
Sequencer
The sequencer application logs are created by prefixing the path specified by the environment variables $RT_REDIRECT_OUT and $RT_REDIRECT_ERR. These are set in Kubernetes via the charts/operator. The default prefix specified by these environment variables is /tmp/rt_log.