Send Feedback
Skip to content

Securing KDB-X Connections with SSL/TLS

This guide explains how to set up, configure, and verify TLS-encrypted connections between KDB-X server and client processes for local development. For general information, refer to the [interprocess communication (IPC)] documentation](ipc.md).

Overview

This guide covers the following topics for securing KDB-X connections:

Prerequisites

Before you can configure TLS, you must install the OpenSSL libraries and ensure that KDB-X can locate them on your system. For further details, refer to the official OpenSSL documentation.

Install OpenSSL

First, ensure that your operating system has the latest OpenSSL libraries installed. Then, add the library path to the appropriate environment variable for your system:

  • Linux: LD_LIBRARY_PATH
  • macOS: DYLD_LIBRARY_PATH
  • Windows: PATH

Verify the OpenSSL version

KDB-X automatically searches for compatible OpenSSL shared libraries on your system. OpenSSL 1.1 is supported since v4.0 2020.03.17.

Since v4.1t 2022.03.25, KDB-X attempts to load versioned shared libraries in the following order:

libssl.so, libssl.so.3, libssl.so.1.1, libssl.so.1.0, libssl.so.10, libssl.so.1.0.2, libssl.so.1.0.1, libssl.so.1.0.0

libssl.3.dylib, libssl.1.1.dylib

KDB-X loads both libssl and libcrypto. The library names are prioritized as follows:

Architecture libssl libcrypto
w64 libssl-3-x64.dll
libssl-1_1-x64.dll		 libcrypto-3-x64.dll
libcrypto-1_1-x64.dll
w32 libssl-3.dll
libssl-1_1.dll		 libcrypto-3.dll
libcrypto-1_1.dll

Windows installation

The Windows build was tested with the pre-compiled libraries from Win32/Win64 OpenSSL. You must also install the Microsoft Visual C++ redistributable package from Microsoft's website.

Before v4.1t 2022.03.25, KDB-X loaded the following unversioned files:

libssl.so

libssl.dylib

KDB-X loads both libssl and libcrypto. The library names are prioritized as follows:

Architecture libssl libcrypto
w64 libssl-1_1-x64.dll libcrypto-1_1-x64.dll
w32 libssl-1_1.dll libcrypto-1_1.dll

How-to guide: basic TLS setup

This section guides you through creating self-signed certificates and configuring a server and client for a secure TLS connection. This setup is ideal for a local development environment.

Step 1: Generate certificates

TLS requires a server to have a certificate and a private key. For local testing, you can generate your own self-signed certificates using the openssl command-line tool.

The following script creates a local Certificate Authority (CA) and uses it to sign certificates for both a server and a client. For more information, refer to the OpenSSL documentation.

# Create a dedicated directory for your certificates
$ mkdir $HOME/certs && cd $HOME/certs

# --- 1. Create the Certificate Authority (CA) ---
# Create a private key for your CA. Keep this secret.
$ openssl genrsa -out ca-private-key.pem 2048
# Create a self-signed root certificate for your CA. This is your trust anchor.
$ openssl req -x509 -new -nodes -key ca-private-key.pem -sha256 -days 365 \
  -out ca-cert.pem -subj "/C=US/ST=CA/L=Somewhere/O=Someone/CN=MyTestCA"

# --- 2. Create the Server Certificate ---
# Create a private key for the server.
$ openssl genrsa -out server-private-key.pem 2048
# Create a Certificate Signing Request (CSR) for the server.
$ openssl req -new -sha256 -key server-private-key.pem \
  -subj "/C=US/ST=CA/L=Somewhere/O=Someone/CN=MyTestServer" -out server.csr
# Use your CA to sign the server's CSR, creating the final certificate.
$ openssl x509 -req -in server.csr -CA ca-cert.pem -CAkey ca-private-key.pem \
  -CAcreateserial -out server-cert.pem -days 365 -sha256

# --- 3. Create the Client Certificate ---
# Create a private key for the client.
$ openssl genrsa -out client-private-key.pem 2048
# Create a CSR for the client.
$ openssl req -new -sha256 -key client-private-key.pem \
  -subj "/C=US/ST=CA/L=Somewhere/O=Someone/CN=MyTestClient" -out client.csr
# Use your CA to sign the client's CSR, creating the final certificate.
$ openssl x509 -req -in client.csr -CA ca-cert.pem -CAkey ca-private-key.pem \
  -CAcreateserial -out client-cert.pem -days 365 -sha256

To check the contents of the generated certificates, run:

# View the server Certificate Signing Request (CSR)
$ openssl req -in server.csr -noout -text
# View the server X.509 certificate
$ openssl x509 -in server-cert.pem -noout -text
# View the client Certificate Signing Request (CSR)
$ openssl req -in client.csr -noout -text
# View the client X.509 certificate
$ openssl x509 -in client-cert.pem -noout -text

To verify the generated certificates against the Certificate Authority (CA), run:

# Verify the server certificate
$ openssl verify -CAfile ca-cert.pem server-cert.pem
# Verify the client certificate
$ openssl verify -CAfile ca-cert.pem client-cert.pem

Simpler certificate generation

For local development, consider using mkcert. It is a simple tool for making locally-trusted certificates without the complexity of openssl commands.

Secure your certificates

Store your certificates and private keys outside of directories accessible from within KDB-X. If a remote user can access your KDB-X home directory, they could potentially steal your server's private key file.

Step 2: Configure and start the server

Set the environment variables to point to the server's certificate, private key, and the CA certificate used to verify connecting clients.

# Set the path to the server's public certificate
$ export SSL_CERT_FILE=$HOME/certs/server-cert.pem
# Set the path to the server's private key
$ export SSL_KEY_FILE=$HOME/certs/server-private-key.pem
# Set the path to the CA certificate for verifying client certificates
$ export SSL_CA_CERT_FILE=$HOME/certs/ca-cert.pem

Start KDB-X with the -E command-line option to enable TLS Server Mode. For more information about configuration, refer to the listening ports documentation.

# Start a KDB-X server on port 5001 with TLS enabled
$ q -p 5001 -E 1

Step 3: Configure and connect the client

In a new terminal, set the environment variables for the client. To support mutual authentication, the client needs its own certificate and key. It also requires the CA certificate to verify the server's identity.

# Set the path to the client's public certificate
$ export SSL_CERT_FILE=$HOME/certs/client-cert.pem
# Set the path to the client's private key
$ export SSL_KEY_FILE=$HOME/certs/client-private-key.pem
# Set the path to the CA certificate for verifying the server's certificate
$ export SSL_CA_CERT_FILE=$HOME/certs/ca-cert.pem

Start a client KDB-X process and use hopen with the tcps:// scheme to open a secure connection to the server.

The general syntax for a secure handle is:

h:hopen`:tcps://hostname:port[:username:password]

Clients can also request secure HTTP (HTTPS) and WebSockets (WSS) connections.

# Start a client KDB-X process
$ q

/ Open a secure handle to the TLS-enabled server
q)h:hopen`:tcps://localhost:5001

/ Send a command over the encrypted connection
q)h "2+2"
4

Verifying the connection

Use these tools to check your TLS configuration and inspect the details of active connections.

Check the configuration

To view the current TLS settings for a KDB-X process, use the (-26!) utility. It returns a dictionary of the loaded configuration.

q)(-26!)[]
SSLEAY_VERSION   | OpenSSL 3.0.13 30 Jan 2024
SSL_CERT_FILE    | /home/user/certs/server-cert.pem
SSL_CA_CERT_FILE | /home/user/certs/ca-cert.pem
SSL_CA_CERT_PATH | /usr/lib/ssl
SSL_KEY_FILE     | /home/user/certs/server-private-ke..
SSL_CIPHER_LIST  | ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RS..
SSL_VERIFY_CLIENT| NO
SSL_VERIFY_SERVER| YES

Inspect the connection status

To retrieve protocol details for an active connection handle, query the .z.e environment variable on the remote process. It returns a dictionary containing connection information, including whether the peer's certificate was successfully verified.

/ Inspect the connection details for connection handle h
q)h".z.e"
CIPHER  | TLS_AES_256_GCM_SHA384
PROTOCOL| TLSv1.3

Troubleshoot common errors

If you encounter connection issues, check the KDB-X console for error messages from the OpenSSL library. Here are some common examples:

  • No shared cipher: This error occurs if the client and server cannot agree on a cipher suite. Check the SSL_CIPHER_LIST settings on both processes. This results in a 'conn error.

    140735201689680:error:1408A0C1:SSL routines:ssl3_get_client_hello:no
shared cipher:s3_srvr.c:1417:
'conn

  • CA certificate not found: This error indicates that the path specified in SSL_CA_CERT_FILE or SSL_CA_CERT_PATH is incorrect or the file is not accessible. This results in a 'conn error.

    ...error:02001002:system library:fopen:No such file or directory...
...error:2006D080:BIO routines:BIO_new_file:no such file...
...error:0B084002:x509 certificate routines:X509_load_cert_crl_file:system lib...
'conn. OS reports: Protocol not available

Configuration reference: environment variables

You can configure KDB-X's TLS behavior by setting the following environment variables before launching the process. These variables are read at startup and control various aspects of TLS configuration.

KX_ prefix

Since v3.6, KDB-X gives preference to a KX_ prefix for these environment variables (e.g., KX_SSL_CERT_FILE overrides SSL_CERT_FILE). This helps avoid conflicts with other applications that may use the same variable names.

For example, getenvKX_SSL_CERT_FILEhas a higher precedence thangetenv SSL_CERT_FILE for determining config.

KDB-X uses the default path reported by openssl version -d as the base directory for locating certificate files if no absolute path is specified.

$ openssl version -d
OPENSSLDIR: "/opt/local/etc/openssl"

SSL_CERT_FILE

Specifies the path to a file containing the local process's certificate in PEM format. This must be sorted starting with the subject's certificate (actual client or server certificate), followed by intermediate CA certificates if applicable, and ending at the highest level (root).

  • Default: <OPENSSLDIR>/server-crt.pem

SSL_KEY_FILE

Specifies the path to the private key associated with the certificate in SSL_CERT_FILE. This file must also be in PEM format.

  • Default: <OPENSSLDIR>/server-key.pem

SSL_CA_CERT_FILE

Specifies the path to a file containing one or more trusted Certificate Authority (CA) certificates. These CAs are used to verify certificates presented by the peer. The file must be in PEM format and can contain multiple concatenated certificates identified by:

-----BEGIN CERTIFICATE-----
... (CA certificate in base64 encoding) ...
-----END CERTIFICATE-----

Text is allowed before, between, and after the certificates; it can be used, for example, for descriptions of the certificates.

  • Default: <OPENSSLDIR>/cacert.pem

Using public CA bundles

To verify certificates from public websites (e.g., when using .Q.hg with HTTPS), you can import a CA bundle from a reputable source like curl.se.

$ curl https://curl.se/ca/cacert.pem > $HOME/certs/cabundle.pem
$ export SSL_CA_CERT_FILE=$HOME/certs/cabundle.pem

You can also append your own self-signed CA certificates to this file if needed.

SSL_CA_CERT_PATH

Specifies the path to a directory containing individual trusted CA certificate files in PEM format.

  • Default: <OPENSSLDIR>

SSL_CIPHER_LIST

Defines the list of cipher suites that the KDB-X process will offer during the TLS handshake. You can override the default to comply with a specific security policy.

  • Default: The Intermediate compatibility list recommended by Mozilla Server Side TLS.

    ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-
AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-
SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-
AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:
ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:
ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:
DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-
RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-
RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:
AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS

SSL_VERIFY_CLIENT

Controls how a KDB-X server validates a client's certificate.

  • NO (Default): The server does not request a certificate from the client.
  • YES: The server requests a client certificate and will disconnect if the certificate is missing or invalid.
  • REQUESTONLY (Since v4.1t 2024.02.07): The server requests a client certificate but allows the connection to proceed even if the certificate is missing or invalid.
  • IFPRESENT (Since v4.1t 2024.02.07): The server requests a client certificate. It disconnects if an invalid certificate is provided but continues if a certificate is missing entirely.

SSL_VERIFY_SERVER

Controls how a KDB-X client validates the server's certificate.

  • YES (Default): The client verifies the server's certificate against the trusted CAs specified in SSL_CA_CERT_FILE or SSL_CA_CERT_PATH. Expired certificates will fail verification.
  • NO: Disables all certificate verification. This is insecure and should only be used for trusted, private networks.
  • HOSTIP (Since v4.1 2025.11.25): Performs the same verification as YES, and additionally checks that the server's certificate Subject Alternative Name (SAN) or Common Name (CN) matches the hostname or IP address used in the hopen command. Requires OpenSSL 1.0.2 or later.
    • When a hostname is used, it checks if the certificate SAN or CN matches the hostname (See X509_check_host).
    • When an IP is used, it checks if the certificate matches a specified IPv4 (refer to X509_check_ip). For example, connecting with 127.0.0.1 would require the following in the certificate: 
      X509v3 extensions:
        X509v3 Subject Alternative Name:
            IP Address:127.0.0.1

Performance and limitations

Performance considerations

Encrypting connections with TLS introduces overhead to both connection setup and data transfer.

  • Connection setup: A TLS hopen on localhost can be 40-50 times slower than a standard TCP connection due to the certificate exchange and cryptographic handshake process.
  • Data transfer: Once connected, the data transfer overhead is approximately 1.5 times that of a plain connection, assuming your OpenSSL library can leverage AES-NI hardware acceleration.

Because of this overhead, TLS is recommended primarily for long-lived, latency-insensitive, and low-throughput connections.

Threading support

Support for TLS within multi-threaded KDB-X environments has evolved over time.

Before v4.1t 2023.11.10, TLS features were not supported in secondary threads. This limitation applied to:

Since v4.1t 2023.11.10, TLS is supported on any thread where messaging was previously supported, including:

Summary

In this guide, you learned how to:

  • Install and verify the necessary OpenSSL libraries for KDB-X.
  • Generate a self-signed Certificate Authority, server, and client certificates for local development.
  • Configure and launch a KDB-X server and client to communicate over a secure TLS connection.
  • Verify the connection status, inspect cryptographic details, and troubleshoot common errors.
  • Use environment variables to control the TLS behavior of your KDB-X processes.