Send Feedback
Skip to content

Listening port

This document is a reference for configuring KDB-X listening ports for Inter-Process Communication (IPC). A listening port enables a q process to accept inbound connections over TCP/IP or Unix Domain Sockets (UDS). By default, a q process does not listen for connections. This guide covers fundamental port configuration and advanced features such as load balancing and multi-threaded input.

Use the -p command-line option or the \p system command to tell KDB-X to listen to a port. The command-line option and the system command take the same parameters.

\p [rp,][hostname:][portnumber|servicename]
-p [rp,][hostname:](portnumber|servicename)

Where

  • rp (optional) is prefix to enable a specialized load-balancing mode.
  • hostname (optional) is a communication handle that binds the listener to a specific network interface. If omitted, KDB-X listens on all interfaces.
  • portnumber is an integer representing the TCP port.
  • servicename is a service name defined in /etc/services

The process must have the necessary permissions to use the specified port, and the port must be available.

Configuring ports

This section covers the fundamental tasks of starting, checking, and stopping the port listener.

To start listening, provide a port number. Where no parameter is specified in the system command, the listening port is reported.

q)\p 81
q)\p
81i

The default is 0i. A result of 0i indicates that the process is not listening.

q)\p
0i

To stop the process from listening for new connections, set the port to 0.

q)\p 0

Advanced port configuration

Beyond a single integer, KDB-X offers several ways to specify a listening port.

Service names

Given a servicename, q will look up its port number in /etc/services.

q)\p commplex-main  / servicename
q)\p
5000i

If you know the process is for clients on the localhost only, choose localhost:port for maximum security.

Ephemeral port

A portnumber of 0W means pick a random available port within the range 32768–60999.

q)\p 5010     / set port 5010
q)\p
5010i
q)\p 0W       / pick a random available port within the range 32768 - 60999
q)\p
45512i
q)\p 0        / turn off listening port

Port range

You can specify an inclusive range of ports. KDB-X will randomly try ports within this range until it finds an available one to bind to. This feature is available since V3.5/3.6 2023.03.13, V4.0 2022.10.26, V4.1 2022.11.01`.

q)\p 2000/2010            / use a free port between 2000 and 2010
q)\p
2003i

Specialized port modes

These modes alter the underlying architecture of how KDB-X handles incoming connections.

Load balancing

Optional parameter rp enables the use of the SO_REUSEPORT socket option, which is available in newer versions of many operating systems, including Linux (kernel version 3.9 and later). This socket option allows multiple sockets (KDB-X processes) to listen on the same IP address and port combination. The kernel then load-balances incoming connections across the processes. (Since V3.5.)

Via the command line when starting a KDB-X process:

$ q -p rp,5000

Or from within the q session:

q)\p rp,5000

Socket sharding with KDB-X and Linux

A load-balancing KDB-X server

Multi-threaded input mode

A negative port sets a multi-threaded port. This mode must be set at startup and cannot be changed at runtime, i.e. do not dynamically switch between positive port and negative port.

When active, each IPC connection will create a new thread for its sole use. It is designed for serving in-memory static data to an externally constrained number of clients. It is not intended for use as a gateway, or serving mutable data.

Each connection uses its own heap with a minimum of 64MB, the real amount depending on the working space required by the query being executed. \ts can be used to find the memory requirement of a query.

$ q -p -5001\5010

Restrictions in multi-threaded input mode:

  • queries on connection threads are unable to update globals (will signal a 'no update error)
  • .z.po is not called on connect
  • .z.pc is not called on disconnect
  • .z.W has a view on main thread sockets only
  • Cannot send async message
  • Views can be recalculated from the main thread only
  • Uncompressed memory pages will not be shared between threads (i.e. same situation as with starting a separate hdb for each request)

Main thread considerations:

The main thread remains responsible for console input, script execution, timer .z.ts on timer expiry, and can update global variables. Updates from the main thread use a multiple-read/single-write lock, which blocks new queries and waits for existing ones to complete. This means that while the main thread is processing an update, none of the connection threads will be processing any input.

Any connections made via IPC from the main thread, can be monitored for callbacks (for example via an async callback) which in turn can update globals.

Updates can impact performance and should not be frequent.

Feature support:

  • HTTP/WebSockets: Supported since 4.1t 2021.03.30 (without TLS). A custom .z.ph which does not update global state should be used with HTTP.
  • Outgoing Sockets: One-shot synchronous requests and HTTP client requests are permitted, but they can be inefficient as a new socket is opened and closed for each request. Other socket usage is blocked and will signal a 'nosocket error (TLS/SSL support added in 4.1t 2023.11.10).

In multithreaded input mode, the seed for the random-number generator used for threads other than the main thread is based on the socket descriptor for that connection; these threads are transient – destroyed when the socket is closed, and no context is carried over for new threads/connections.

Unix domain sockets (UDS)

In additions to listening on TCP port 5000, setting the listening port with -p 5000 also creates a UDS (Unix domain socket) on /tmp/kx.5000. You can disable listening on the UDS, or change the default path from /tmp using environment variable QUDSPATH.

You can control UDS behavior with the QUDSPATH environment variable before setting the port:

q)/ disable listening on unix domain socket
q)system"p 0";setenv[`QUDSPATH;""];system"p 6000"
q)/ use /home/kdbuser as path
q)system"p 0";setenv[`QUDSPATH;"/home/kdbuser"];system"p 6000"

To connect to a process via its UDS: 

q)hopen `:unix://5000

Note on compatibility

Since V3.5, KDB-X on Linux uses an abstract UDS namespace to avoid file permission issues. This makes V3.5+ incompatible with V3.4 for UDS connections.

On macOS:

q)\p 5000
q)\ls /tmp/kx*
"/tmp/kx.5000"
q)system"p 0";setenv[`QUDSPATH;""];system"p 5000"
q)\ls /tmp/kx*
ls: /tmp/kx*: No such file or directory
'os
q)system"p 0";setenv[`QUDSPATH;"/tmp/kxuds"];system"p 5000"
'cannot listen on uds /tmp/kxuds/kx.5000. OS reports: No such file or directory
  [0]  system"p 0";setenv[`QUDSPATH;"/tmp/kxuds"];system"p 5000"
                                                  ^
q)\mkdir /tmp/kxuds
q)system"p 0";setenv[`QUDSPATH;"/tmp/kxuds"];system"p 5000"
q)\ls /tmp/kxuds
"kx.5000"

Security

Opening a port makes your KDB-X process accessible over the network, including to HTTP requests. It is critical to secure any process with an open port.

In a production environment secure any process with an open port.

hopen

Command-line options -E, -p; system command \p