RT Stream log files
When diagnosing issues with a data pipeline that uses an RT cluster (known in the UI as a KX Insights Stream) it is helpful to know where the stream logs reside. This is intended to assist you when the data does not flow to database, allowing you to find the files and identify how far the messages have travelled through the pipeline.
Stream log files
Each publisher and subscriber writes and reads from a set of RT log files in a directory representing a stream of messages.
Publishers and subscribers specify a 64 bit position in the stream when writing or reading from it. That position first identifies the log file which contains that message, with RT log files being named as
- x is the top 20 bits of the position
- y is the next 14 bits of the position
The remaining 30 bits then locate the start of the message in the correct log file. Therefore, each log file has a maximum size of 1GB before automatically rolling to the next log file.
|ODBC driver on Linux||/tmp/rt|
|Java SDK on Linux||KXI_LOG_PATH environment variable|
|ODBC driver or Java SDK on Windows||Path defined in HKEY_CURRENT_USER\SOFTWARE\kx\kodbcDriver\LogPath|
|Inbound RT Stream files||/s/in/$TOPIC|
|Outbound RT Stream files||/s/out/OUT|