Send Feedback
Skip to content

Use WebSockets with KDB-X

This page explains how to implement WebSocket servers and clients in KDB-X for real-time, bi-directional communication. For more information on WebSockets in q, see the I/O chapter in Q for Mortals.

Overview

KDB-X supports the WebSocket protocol (since v3.0), providing a persistent, bi-directional, full-duplex communication channel over a single TCP connection. WebSockets allow the server to push data to the client immediately as it becomes available. This makes them ideal for interactive, real-time applications such as trading dashboards, live feeds, and chat systems.

This guide explains how to:

Create a WebSocket server

Configure a KDB-X process as a WebSocket server by listening on a port and defining the .z.ws message handling function. The server executes this callback for every message received from a client.

Example: Echo server

Create a simple server that echoes any message it receives back to the sender.

  1. Start a q session and listen on port 5000. 

    q)\p 5000

  2. Define the .z.ws callback. Use neg[.z.w] to send the incoming message x back to the client associated with the current connection handle .z.w. 

    // .z.ws: a global callback for WebSocket messages
// x: the incoming message from the client
// .z.w: the connection handle of the current client
// neg: sends a message asynchronously (required for WebSockets)
q).z.ws:{neg[.z.w] x}

Test this server using any WebSocket-capable client. For example, open the ws.htm sample client in a browser and connect to ws://localhost:5000. Any message you send will be echoed back in the output area.

Manage your session

The examples in this guide use port 5000. If you run multiple server examples, ensure you close the previous q process or choose a different port to avoid "address in use" errors.

Example: Remote execution server

Redefine .z.ws to evaluate received messages as q code. Use .Q.s to serialize the result.

// .Q.s: serializes the result into a byte vector suitable for IPC
// value: evaluates a string as a q expression
q).z.ws:{neg[.z.w] .Q.s value x}

Security Risk

This example executes arbitrary code from any connected client. Never expose raw value or evaluation capabilities to untrusted clients in production.

Instead, use a Command Pattern: parse the input, validate the command against an allowlist, and dispatch to specific handler functions.

Use a trap (@) to catch and return potential errors in the submitted q code gracefully.

// @[function; args; error_handler]
// If `value x` fails, the error handler formats the error message
q).z.ws:{neg[.z.w] @[.Q.s value@;x;{"'",x,"\n"}]}

Handle WebSocket events

KDB-X uses special callbacks in the .z namespace to manage the WebSocket lifecycle.

  • .z.ws: (WebSocket message) The primary callback for handling incoming messages from a client
  • .z.wo: (WebSocket open) A callback executed when a client opens a new connection. It receives the connection handle as its argument
  • .z.wc: (WebSocket close) A callback executed when a client connection is closed. It receives the connection handle as its argument
  • .z.w: (handle) Within a .z callback, this variable holds the integer connection handle for the current client

Define .z.wo and .z.wc to track active connections.

// Create a table to store connection details, keyed by handle
q)activeWSConnections:([handle:()] connectTime:())

// On open, add the new connection to the table
// The argument 'x' is the connection handle passed by the system
q).z.wo:{`activeWSConnections upsert (x;.z.t)}

// On close, remove the connection from the table
q).z.wc:{delete from `activeWSConnections where handle=x}

View all connections

View all current socket connections, including WebSockets, with the internal function -38!.

Send data and manage connections

Send data to any WebSocket client asynchronously by using the negative of the connection handle (neg[h]).

// Send a message to a single client
q)neg[h] "Hello, client!"

// Broadcast the same message to multiple clients
// Use @\: to apply the message to each handle in the list
q)handles:exec handle from activeWSConnections
q)neg[handles]@\:"Market is now open"

Close connections

Close unneeded connections using hclose. To ensure all pending outgoing messages are sent before closing, flush the connection first.

// Assume 'h' is a valid connection handle
q)neg[h][]  / Flush pending data (blocks until sent)
q)hclose h  / Close the connection handle

To gracefully close all active WebSocket connections:

// Flush and close all connections in the tracking table
q){neg[x][];hclose x} each exec handle from activeWSConnections

Create a WebSocket client

A KDB-X process can act as a WebSocket client to connect to other servers (since v3.2t).

.z.ws serves a dual purpose

The .z.ws callback handles incoming messages whether your process is acting as a server or a client. When acting as a client, .z.ws receives messages sent by the remote server.

Define .z.ws first

You must define the .z.ws callback before initiating a connection. Failing to do so results in a '.z.ws undefined error.

Open a connection by sending an HTTP Upgrade request to the target server.

// 1. Define .z.ws to handle incoming messages (required)
q).z.ws:{0N!x}

// 2. Send an HTTP GET request with the required "Upgrade" headers
// `$":ws://host:port"` creates a connection handle for the request
q)h:(`$":ws://127.0.0.1:5000")"GET / HTTP/1.1\r\nHost: 127.0.0.1:5000\r\n\r\n"

A successful connection returns a two-item list containing the integer connection handle and the HTTP 101 ("Switching Protocols") response from the server.

// Expected output on success
q)h
(6i;"HTTP/1.1 101 Switching Protocols\r\nConnection: Upgrade\r\nUpgrade: websocket\r\nSec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk=\r\n\r\n")

If the server denies the upgrade request, the handle is a null integer (0Ni).

// Expected output on failure (for example, bad request)
(0Ni;"HTTP/1.1 400 Bad Request\r\n...")

Other connection issues, such as a non-existent host, signal a standard q error.

Example: Client and server

This example demonstrates a KDB-X client communicating with a KDB-X server.

  1. Server: Start a q process on port 5000. Define .z.ws to format and return a reply string. 

    q)\p 5000
q).z.ws:{neg[.z.w] "server replied with ",$[10=type x;x;raze string x];}

  2. Client: Define .z.ws to print messages received from the server. Then, connect to the server. 

    q).z.ws:{0N!"Client Received Msg:";0N!x;}
q)r:(`$":ws://127.0.0.1:5000")"GET / HTTP/1.1\r\nHost: 127.0.0.1:5000\r\n\r\n"

  3. Client: Send messages to the server using the handle from the connection response (r 0). 

    q)neg[r 0]"test"      / Send a string
q)neg[r 0]0x010203    / Send a byte vector

    The client terminal displays the replies received from the server. 

    Client Received Msg:
"server replied with test"
Client Received Msg:
"server replied with 010203"

  4. Client: Close the connection. Use the handle (the first item of the list returned by the connection request). 

    q)hclose r 0

Advanced topics

Authentication and security

Server-side

The initial WebSocket connection is an HTTP Upgrade request. Secure this handshake using the .z.ac (HTTP auth) callback, which allows integration with standard authentication mechanisms like LDAP, OAuth2, or OpenID Connect.

Client-side

To authenticate your client, customize the HTTP Upgrade request header with credentials, such as a bearer token or a session cookie.

For Basic access authentication, include the username and password in the connection string: 

q)`:ws://username:password@127.0.0.1:5001 "GET / HTTP/1.1\r\nHost: 127.0.0.1:5001\r\n\r\n"

Secure connections (SSL/TLS)

To establish a secure WebSocket connection (wss://), first configure your KDB-X process for SSL/TLS. Then, replace the ws:// prefix with wss:// in the connection string.

Use an SSL/TLS wrapper

Alternatively, use a tool like stunnel to provide a secure wrapper for a standard WebSocket server. This approach decouples SSL/TLS termination from the KDB-X process.

Data serialization

JSON

JSON is the standard format for web communication and is natively supported by KDB-X. Use .j.k to parse incoming JSON strings into KDB-X dictionaries, and .j.j to serialize KDB-X data into JSON strings.

// Example: Parse JSON, extract "cmd", and return a JSON confirmation
.z.ws:{
  d:.j.k x;                              / Parse incoming JSON string to dictionary
  res:`cmd`status!(d`cmd;"received");    / Create response dictionary
  neg[.z.w] .j.j res                     / Serialize to JSON and send
 }

KDB-X IPC format (c.js)

For higher performance with KDB-X specific data types, you can send binary serialized data. The client must be able to deserialize this format (for example, using the c.js library in a browser).

In KDB-X, use -8! (to bytes) to encode data before sending and -9! (from bytes) to decode received data.

Example: wslogin.htm

The wslogin.htm example demonstrates sending a JavaScript dictionary to KDB-X using c.js.

  1. Start the KDB-X server on port 5000. The .z.ws function decodes the incoming byte vector, extracts its values, and sends the values back in serialized form. 

    q)\p 5000
// -9!x:      Deserialize incoming byte vector 'x' into a KDB-X object
// value ...: Get the values from the resulting dictionary
// -8!...:    Serialize the result back into a byte vector for sending
q).z.ws:{neg[.z.w] -8!value -9!x;}

  2. Open wslogin.htm in a browser (ensure c.js is in the same directory).

  3. Click login. The browser serializes a JavaScript dictionary ({u:"user", p:"At0mbang."}) and sends it.

  4. The server receives the byte vector, deserializes it into a KDB-X dictionary (`u`p!("user";"At0mbang.")), and returns the serialized values (("user";"At0mbang.")) to the browser.

Performance and encoding

Compression

KDB-X supports permessage-deflate WebSocket compression, which reduces network traffic. Compression is automatically negotiated during the connection handshake. In Chrome DevTools, a successful negotiation includes the Sec-WebSocket-Extensions: permessage-deflate header in the HTTP response.

You can measure the impact of compression using \ts (time and space). Compressed data takes longer to queue (higher CPU usage) but consumes less bandwidth.

// Generate compressible data
q)v:10000#.Q.a 

// Queue 1000 messages to an existing handle (uncompressed)
// Note: Faster to queue, but consumes more memory/bandwidth
q)\ts do[1000;(-5)v];show sum each .z.W
5| 10004000
6| 0
14 20610976

// Queue 1000 messages to a handle requesting compression
// Note: Slower to queue (CPU cost), but significantly smaller payload
q)\ts do[1000;(-6)v];show sum each .z.W
5| 0
6| 47022
94 4354944

Disable compression

To disable compression for a specific client connection (for example, to prioritize lower CPU usage over bandwidth), send the sec-websocket-protocol HTTP header with the value kxnodeflate.

// Example JavaScript client disabling compression
let ws = new WebSocket(url, "kxnodeflate");

UTF-8 encoding

Encoding requirement

The WebSocket protocol requires all text messages to be UTF-8 encoded. Attempting to send text with invalid encoding signals a 'utf8 error in KDB-X.

Example: Real-time data publisher

This example demonstrates a simple publisher-subscriber model where a KDB-X process pushes real-time trade and quote updates to a web browser.

Setup

  1. Download the project files from kxcontrib/websocket.
  2. Run q pubsub.q to start the publisher server.
  3. Run q fh.q to start a feed handler that generates dummy data.
  4. Open websockets.html in your browser to view the real-time updates.

Server: Subscription management

This application allows the web client to subscribe to specific server-side functions. The server's .z.ws handler evaluates any incoming message, allowing the client to call functions like sub.

// pubsub.q: Let the client call functions directly
.z.ws:{value x}

A subs table tracks active subscriptions, storing the client handle, the subscribed function, and its arguments.

// subs table schema: handle, function, params
q)subs:2!flip `handle`func`params!"is*"$\:()

When a client connects, it sends a message to call the sub function, which registers its subscription in the subs table.

// Define the subscription function
// x: function name to subscribe to
// y: parameters for the function
sub:{`subs upsert(.z.w;x;enlist y)}

Server: Publishing logic

A timer on the server (.z.ts) periodically calls the pub function for each subscription. pub executes the subscribed function and pushes the result to the client. The data is serialized to JSON using .j.j before being sent.

// pubsub.q
pub:{
  row:(0!subs)x;
  // Execute the subscribed function (for example, getTrades`AAPL`GOOG)
  newData:(value row`func)row`params;
  // Send update to client as a JSON string
  (neg row`handle) .j.j `func`result!(row`func;newData);
  }

Set up a timer to publish updates for all subscriptions every second:

.z.ts:{pub each til count subs}
\t 1000

Client: Handling updates

On the client side, the JavaScript onmessage handler parses the incoming JSON string. To route the data correctly (for example, to a quotes table or a trades table), the server's response includes a unique identifier. Here, the name of the subscribed function (func) serves as this identifier.

// websockets.html
ws.onmessage = function(e) {
    // Parse the JSON string from the server
    var d = JSON.parse(e.data);

    // Use the 'func' property to call the correct UI update handler
    switch(d.func){
        case 'getSyms'   : setSyms(d.result);   break;
        case 'getQuotes' : setQuotes(d.result); break;
        case 'getTrades' : setTrades(d.result); break;
    }
};

This architecture creates a simple but effective real-time web application, showcasing how to integrate KDB-X and WebSockets to deliver live data.

Summary

This guide explained how to:

  • Configure a KDB-X process as a WebSocket server with .z.ws
  • Implement a WebSocket client in KDB-X
  • Manage the connection lifecycle with event handlers like .z.wo and .z.wc
  • Secure connections and serialize data with JavaScript and JSON
  • Build a real-time data publisher application