Skip to content

RT Configuration

This section describes some the configuration details in RT.

RT service ports

The port that RT services run on can be configured via environment variables. If the environment variable is not set, then the default is used.

env var default notes
RT_SUP_ADMIN 6001 RT supervisor which launches RT services
RT_REST_TCP 6000 RT service which responds to REST requests
RT_PUSH_SERVER 5000 Replicator that clients publish to with encryption
RT_PUSH_SERVER_INT 5001 Replicator that clients publish to
RT_SUB_SERVER 5003 Replicator that clients subscribe from. Supports topic filtering
RT_SUB_SERVER_EXT 5006 Replicator that clients subscribe from with encryption. Supports topic filtering
RT_PULL_SERVER 5002 Replicator that clients subscribe from. Does not support topic filtering
RT_PULL_SERVER_EXT 5007 Replicator that clients subscribe from with encryption. Does not support topic filtering
RT_AWS_BACKUP_ADMIN 4997 Optional service to backup RT logs to AWS
RT_AZURE_BACKUP_ADMIN 4996 Optional service to backup RT logs to Azure
RT_GCS_BACKUP_ADMIN 4995 Optional service to backup RT logs to GCS

Sequencer Endpoints

Internal Publisher

An internal publisher starts a set of push_client replicators which connect to each of the internal push_server replicators running in the RT sequencers using the endpoints:

  • ${RT_TOPIC_PREFIX}<streamid>-0:5002
  • ${RT_TOPIC_PREFIX}<streamid>-1:5002
  • ${RT_TOPIC_PREFIX}<streamid>-2:5002

Note

${RT_TOPIC_PREFIX} is defaulted as follows: - Docker is set to rt- - Kubernetes is set to the release name of the RT chart - kdb Insights Enterprise is set to rt-

For example: in kdb Insights Enterprise if the streamid is set to data the endpoints will be rt-data-0:5002, rt-data-1:5002, rt-data-2:5002.

Internal Subscriber

An internal subscriber starts a set of pull_client replicators which connect to each of the internal pull_server replicators running in the RT sequencers using the endpoints:

Note

RT introduced a new subscription service, the sub_server which supports topic filtering. We would encourage users to take advantage of this feature over the pull_client. However, if the original pull_client is required, the port below can be changed to 5001 from 5002

setenv[RT_PORT] "5003"
  • ${RT_TOPIC_PREFIX}<streamid>-0:${RT_PORT}
  • ${RT_TOPIC_PREFIX}<streamid>-1:${RT_PORT}
  • ${RT_TOPIC_PREFIX}<streamid>-2:${RT_PORT}

Note

${RT_TOPIC_PREFIX} is defaulted as follows: - Docker is set to rt-. - Kubernetes is set to the release name of the RT chart. - kdb Insights Enterprise is set to rt-

For example: in kdb Insights Enterprise if the streamid is set to data the endpoints will be rt-data-0:5001, rt-data-1:5001, rt-data-2:5001.

External Publisher

External publishers use encryption with mutable TLS. An external publisher starts a set of push_client replicators with SSL enabled which connect via load balancers to each of the external push_server replicators running in the RT sequencers using the endpoints:

  • <load_balancer_1>:5000
  • <load_balancer_2>:5000
  • <load_balancer_3>:5000

The mechanism for determining the load balancer endpoints and certificate exchange varies depending on the deployment type:

External subscriber

External subscribers use encryption with mutable TLS. An external subscriber uses the sub_client library with SSL enabled which connects to each of the external sub_server replicators running in the RT sequencers using the endpoints:

kdb Insights Enterprise does not provide load balancer endpoints for subscription. Port forwarding, or some equivalent method, would be required to expose the host endpoints.

  • hostname-1:5006
  • hostname-2:5006

  • hostname-3:5006

  • The Docker demonstration uses docker to look up port mappings with pre-generated certs
  • The Kubernetes demonstration uses kubectl to look up the load balancer endpoints with pre-generated certs

Stream log files

Each publisher and subscriber writes and reads from a set of RT stream log files in a directory.

Publishers and subscribers specify a 64 bit position in the stream when writing or reading from it. That position first identifies the log file which contains that message, with RT log files being named as log.x.y where:

  • x is the top 20 bits of the position
  • y is the next 14 bits of the position

The remaining 30 bits then locate the start of the message in the corresponding log file. Therefore, each log file has a maximum size of 1GB before automatically rolling to the next log file.

Additional files created by an RT stream. You should not be modify nor delete these files:

  • .session file.
  • state.X file

RT Log Configuration

Log levels are controlled by RT_LOGLEVEL_CONSOLE. However, there are some distinct services, such as QuRaft (responsible for RAFT) and the RT replicators, which have their own log levels

Server

  1. RT_LOGLEVEL_CONSOLE, Sets the RT server logging level. Can be one of ERROR, INFO, DEBUG, TRACE. Default INFO.
  2. RT_QURAFT_LOG_LEVEL, Sets the Raft logging level. Can be one of FATAL, ERROR, INFO, DEBUG, TRACE, OFF. Default INFO.
  3. REPLICATOR_LOGLEVEL_CONSOLE, sets the pull and push server replicator logging (pull_server and push_server)
  4. RT_SUBSERVER_LOGLEVEL, sets the sub_server replicator logging

Client

  1. RT_LOGLEVEL_CONSOLE, Sets the RT client logging level. Can be one of ERROR, INFO, DEBUG, TRACE. Default INFO.

The replicator log level can be controlled via the parameter repl_console_log_level. Please see the relevant interface docs for more information on setting this.

Sub server caching

The RT sub_server service can cache information to reduce the disk I/O of your system.

The sub_server will cache your message headers and optionally cache the message payload. The size of the cache is controlled by the environment variables below.

Note

The caching offers some flexibility for disk I/O. However, higher cache sizes will mean that more RAM required by your system

env var default notes
RT_SUBSERVER_HEADER_CACHE_SIZE 5000 Number of message headers that the cache can grow to. Each message header requires approx 450 bytes
RT_SUBSERVER_HEADER_CACHE_PURGE_INTERVAL 1000 Frequency in milliseconds between items being considered for clearing from the cache
RT_SUBSERVER_PAYLOAD_CACHE_SIZE 250 Number of message payloads that the cache can grow to. Each message payload requires up to 60kb
RT_SUBSERVER_PAYLOAD_CACHE_PURGE_INTERVAL 1000 Frequency in milliseconds between items being considered for clearing from the cache