Routing configuration

Query routing is managed by two primary components, the Service Gateway, and the Resource Coordinator. Queries are broken into partials and distributed across query instances and collected into a final result in the Aggregator

process diagram

Unless otherwise specified, the arrows in the diagram represent asynchronous q IPC communication between processes. Query flow is as follows.

arrow	description
`1`	Client makes API request to a Service Gateway replica. This can be sync or asynchronous.
`2`	The Service Gateway forwards the request to the Resource Coordinator.
`3`	The Resource Coordinator sends partial requests to each DAP relevant to the query based on purview.
`4`	DAPs forward their responses to a single Aggregator for aggregation.
`5`	The Aggregator sends the response to the same Service Gateway the client connected to.
`6`	The Service Gateway sends response back to client.

These components route queries across one or more assemblies and are installed as part of a base deployment. These components are generally configured using using environment variables or a values file if using a Kubernetes based deployment. In both environment, environment variables can be used to change configuration.

Environment Variables

Configuration can be supplied to the Service Gateway or the Resource Coordinator using environment variables. Environment variables are configured differently depending on the method of deployment. In all cases, the variables are always string values.

DockerKubernetes

In Docker, environment variables are supplied using under an environment key for the target service as a list of key-value pairs.

services:
  sg:
    environment:
      - KXI_NAME=sg

In a Kubernetes deployment, environment variables are passed to compnents using a values file. For the Service Gateway, these values are supplied under an sg-gateway key. For the Resource Coordinator, these are supplied under a resource-coordaintor key.

kdb Insights Enterprise

When running in an enterprise deployment, values are nested under either a service-gateway or qe-gateway key in the values file. See enterprise deployment configuration for more details.

service-gateway:
  sg-gateway:
    env:
      KXI_SG_TIMEOUT: "50000"
  resource-coordinator:
    env:
      KXI_SG_TIMEOUT: "50000"

Service Gateway

name	description
`GATEWAY_QIPC_PORT`	Gateway port for QIPC traffic.
`GATEWAY_HTTP_PORT`	Gateway port for HTTP traffic.
`DISCOVERY_PROXY`	Discovery proxy address (not required if not using discovery).
`KXI_SG_RC_ADDR`	`host:port` of RC process to connect to.
`KXI_SG_TIMEOUT`	Default request timeout in milliseconds (default: `60000`).
`KXI_SG_TIMEOUT_MARGIN`	Controls the amount of time to wait for the RC to complete a request in milliseconds. The service gateway waits this many milliseconds after a request has timeout out for the RC to issue a timeout request. (default: `30000`)
`KXI_RC_SERVICE_NAME`	The key that the gateway looks for to find Resource Coordinators within the discovery registry. Default is `KXI-SG-RC`.
`KXI_SG_RC_LABEL_SELECTOR`	Allow the gateway to find Resource Coordinators by label selector. Required for Kubernetes-based discovery.
`KXI_SG_DA_LABEL_SELECTOR`	Allow the gateway to find data-access pods by label selector.
`KXI_SG_BUFFER_INITIAL`	Initial size in bytes to allocate per connected aggregator (default: 2 MB).
`KXI_SG_BUFFER_RETAIN`	Maximum size in bytes to hold on to per connected aggregator (default: 2 MB).
`KXI_AUTH_DISABLED`	Set `KXI_AUTH_DISABLED=0` to use the gateway with Keycloak.
`KXI_OIDC_JSON_PATH`	OIDC JSON used for mapping Keycloak authentication endpoints.

Resource Coordinator

name	description
`KXI_NAME`	Process name.
`KXI_PORT`	Port.
`KXI_SG_RC_ADDR`	`host:port` of RC process to connect to.
`KXI_GC_FREQ`	Frequency in milliseconds to run garbage collect in a timer (default: `600000`, set to `0` to disable).
`KXI_SG_TIMEOUT`	Default request timeout in milliseconds (default: `60000`).
`KXI_SG_DISC`	Multi-RC discovery mode.
`KXI_SG_CONN_TIMEOUT`	Timeout on connection open.
`KXI_SG_MAX_RETRY`	Maximum number of retries the RC is willing to do for retry-able errors.
`KXI_ALLOWED_SBX_APIS`	Comma-delimited list of sandbox APIs to allow in non-sandbox RCs (e.g. `.kxi.sql,.kxi.qsql`).
`KXI_SG_REQ_DEL_FREQ`	Time in milliseconds to run the request table delete routine (default: `10000`).
`KXI_SG_QUEUE_DEL_FREQ`	Time in milliseconds to run the queue table delete routine (default: `10000`).
`KXI_SAPI_HB_FREQ`	Time in milliseconds to run the heartbeat to connected processes (default: `30000`).
`KXI_SAPI_HB_TOL`	Number of heartbeat intervals a process can miss before being disconnected (default: `2`).
`KXI_SG_MULTI_RC_TO_MAX_WAIT`	On a timeout of a multi-RC request, number of 10 second iterations to wait for timeout details from all RCs (default: 1).
`KXI_ENABLE_FLUSH`	Set to `true` to enable async flush on messages from RC to DA/Agg and from Agg to GW (default `false`).