Interprocess communication basics¶

This page covers the fundamentals of interprocess communication (IPC) in q: establishing and closing connections, and sending synchronous requests.

q processes communicate with each other and external applications using a high-performance TCP/IP-based protocol. This capability is built directly into the language, enabling you to create distributed systems where processes can exchange data and commands efficiently.

Other IPC mechanisms

This document focuses on TCP/IP sockets. q also supports other IPC mechanisms, such as Unix Domain Sockets (UDS), which use the familiar pattern of opening a handle with hopen, sending requests, and closing the connection with hclose.

Opening a port¶

The fundamental IPC workflow involves a server process that listens for incoming connections and a client process that connects to the server to send requests. Before processes can communicate, you must establish a connection. The server must be listening on a port, and the client must open a connection to that port.

To make a q process act as a server, instruct it to listen on a specified port, either at startup or at runtime.

At startup (command line):

# Start a q process listening on port 5001
q -p 5001

At runtime (system command):

// Set the listening port to 5001 from within the q session
q)\p 5001

The operating system's file descriptor limit determines the maximum number of simultaneous connections. If the limit is reached, the server rejects new connections with a 'conn error.

A client process connects to a server using the hopen function, which takes a target host and port. hopen returns an integer handle representing the established connection.

// Connect to a process listening on localhost, port 5001
q)h: hopen 5001

// The handle 'h' is an integer (OS file descriptor)
q)h
3i

A bare integer such as 5001 is shorthand for the local communication handle `::5001 — the host is omitted, so it always targets the local machine. Use the full communication handle syntax to connect to a remote host, over a Unix domain socket, or over SSL/TLS:

q)hopen `:10.43.23.198:5010          / TCP, by IP address
q)hopen `:mydb.us.com:5010           / TCP, by hostname
q)hopen `:mydb.us.com:5010:user:pass / TCP, with credentials
q)hopen `:unix://5010                / Unix domain socket, localhost
q)hopen `:tcps://mydb.us.com:5010    / SSL/TLS, with hostname

Prefer UDS between processes on the same host

When two q processes run on the same host, connect over the Unix domain socket (`:unix://port) rather than TCP (`:port, `::port, or a bare integer). A UDS connection bypasses the network stack, so it uses fewer system resources and gives you lower latency and higher throughput than a TCP connection over loopback.

Sending requests: sync vs. async¶

Once a connection is established, the client can send messages to the server using the connection handle. A message can be either a string to be evaluated or a list representing a function call.

String format: The server parses and evaluates the string.
```
q)h"2+2"
4
```
List format: The server runs the function in the first item of the list with the remaining items as arguments. This is the preferred method for passing data and functions.
```
q)h(+;2;2)
4
```

Using the list format, you can pass local data or even entire functions from the client to the server for evaluation.

// Define a local function on the client
q)clientFn:{4+x}

// Define a variable on the client
q)v:10

// Pass the client's function and variable to the server for execution
q)h(clientFn;v)
14

// Pass an anonymous function (lambda) directly to the server
q)h({x*y};5;10)
50

q supports two primary modes for sending messages: synchronous (request-response) and asynchronous (fire-and-forget). Asynchronous communication is a complex topic and deserves a separate page.

Synchronous requests (request-response)¶

A synchronous request sends a message and blocks until the server processes it and returns a response. This is the default behavior and is ideal for queries that require an immediate result.

By default, the server runs the incoming message using value and automatically sends the result back to the client.

// Send a sync request and wait for the result
q)h"sum 1 2 3"
6

Run remote vs. local functions¶

You can control whether a function defined on the client or a function of the same name on the server runs.

To run a function defined on the server, pass its name as a symbol (for example, `add).
To run a function defined on the client, pass the function value directly (for example, add).

On the server process, define add:

// Server's 'add' function sums its arguments
q)add:{x+y}

On the client process, connect and define a different add:

q)h: hopen `::5001
// Client's 'add' function multiplies its arguments
q)add:{x*y}

// Call the SERVER's function by passing its name as a symbol
q)h(`add;10;5)
15

// Pass the CLIENT's function object to the server for execution
q)h(add;10;5)
50

Avoid nested synchronous requests

Do not nest synchronous requests. For example, if a client sends a synchronous request to a server, and that server then sends another synchronous request to a third process, the responses might arrive out of order, causing unpredictable behavior.

One-shot requests¶

For a single, infrequent query, you can send a synchronous request without explicitly establishing a persistent connection first. However, this approach is less efficient than reusing an existing connection for multiple messages, because each request incurs the overhead of setting up a new connection.

// Shorthand to open a connection, send a query, get a response, and close it
q)`::5001 "1+1"
2

A one-shot request is also the only outgoing socket call permitted from a secondary thread — inside peach, or from a connection thread in multi-threaded input mode. A persistent handle from hopen, and the synchronous or asynchronous requests you send on it, are restricted to the main thread; attempting them from a secondary thread signals a 'nosocket error.

Closing a port¶

To release system resources, close the connection from either the client or server side using hclose when it's no longer needed.

q)hclose h

Explicitly close connections

q does not automatically close connections when their handle goes out of scope in your code. You must explicitly call hclose to close a connection and free up resources on both the client and server.

Callbacks (server-side)¶

On the server side, you can customize how incoming messages are handled by defining callback functions in the .z namespace.

Callback	Trigger	Default behavior
`.z.po`	A new client connection is opened.	Does nothing.
`.z.pc`	A client connection is closed.	Does nothing.
`.z.pg`	A synchronous (get) message is received.	Runs `value x` on the message.
`.z.ps`	An asynchronous (set) message is received.	Runs `value x` on the message.
`.z.pw`	A new client attempts to authenticate.	Validates user credentials.

Override these functions to add custom logic, such as logging, permission checks, or specialized message routing.

Example: Logging incoming requests and connections

// Log connection open events, showing handle, user, and IP
.z.po: {0N!(`ConnectionOpened; `handle`.z.w; `user`.z.u; `ip`.z.a)}

// Log each sync request before executing it
.z.pg: {0N!(`SyncRequest; .z.w; .z.u; x); value x}

// To revert to default behavior, delete your custom definitions
q)\x .z.po .z.pg

Next steps¶

Asynchronous communication — flush the message queue, broadcast to multiple clients, block for a deferred synchronous response, and implement asynchronous callbacks.
Listening Port — configure the port a q process listens on: service names, ephemeral and ranged ports, multi-threaded input mode, load balancing, and Unix domain sockets.
Deferred Response — use -30! to suspend a synchronous reply so a gateway can do asynchronous work before answering the client.
SSL/TLS — encrypt connections between q processes using OpenSSL.
WebSockets — use the WebSocket protocol for persistent, bidirectional communication with browsers and other clients.