Skip to content

Asynchronous communication

This page explains how to send asynchronous interprocess communication (IPC) messages in q: flushing the outbound queue, broadcasting to several clients, writing asynchronous callbacks, and blocking on a handle for an expected reply.

An asynchronous request sends a message and returns at once, without waiting for a response. It suits high-throughput work such as logging and data publishing, and any command whose result you do not need.

To send a message asynchronously, apply it to a negative connection handle, which you get by applying neg to the handle:

q)neg[h]"a:10"                            / tell the server to create a variable 'a'
q)neg[h]("0N!"; "hello from client")       / tell the server to print to its console

Because the client never waits for a response, asynchronous messaging is essential in systems such as tickerplants, where blocking on a slow subscriber would stall every other subscriber.

Size socket buffers for the connection count you expect

Increasing the TCP send and receive buffer sizes in your operating system can improve throughput on a busy connection.

The operating system reserves that buffer memory per socket, whether or not the socket is busy. Scaling the buffers up across thousands of concurrent connections depletes system memory rather than improving throughput, so size them for the number of connections you expect, not for the busiest one.

Flush the message queue

An asynchronous message optimizes for unblocking the sender, not for request latency. By default q buffers the message in user space and writes it to the socket in its own time. An async flush hands the serialized bytes for a handle to the kernel immediately:

q)neg[h][]     / flush every pending async message on handle h
q)neg[h](::)   / the same thing

To inspect the size of the outbound queue, use .z.W for every handle, or -38!x for one handle.

A synchronous message flushes the queue and confirms delivery

Sending any synchronous message on the same handle, such as h"", also flushes the asynchronous queue. Because the remote process handles messages in order, a response to that synchronous message confirms that it has processed every asynchronous message you sent before it.

Broadcast a message

To send one message to several clients, use async broadcast, -25!. It serializes the message once, which makes it more efficient than sending to each client in a loop.

q)(h1; h2; h3): hopen each `::5000`::5001`::5002   / handles for three clients
q)msg: (`someFunction; `arg1; `arg2)
q)-25!((h1; h2; h3); msg)                          / serialize once, send to all three

Asynchronous callbacks

An asynchronous callback lets a client send a request without blocking and still receive the result once the server has it: the server replies by making its own asynchronous call back to a function that the client named in the request. Because neither the request nor the reply blocks, the client can do other work while it waits, and one client's slow or unresponsive callback cannot stall the server's replies to anyone else.

The pattern has three parts: the client's call, the server function's signature, and the server's callback.

  1. The client's call. The client sends an asynchronous request to a named function on the server. Its last argument is a symbol naming a function that exists on the client.

    / (neg h) (`remoteFunc; arg1; ..; argN; `clientCallback)
(neg h) (`proc; 42; `clientFunc)

  2. The server function's signature. Define the remote function to take the callback's name as its final parameter.

    / remoteFunc: {[arg1; ..; argN; callbackName] ... }
proc: {[dataArg; callbackName] ... }

  3. The server's callback. Inside the remote function, .z.w gives the handle of the calling process. Use it to call the client's named function asynchronously, passing the result.

    proc: {[dataArg; callbackName]
  result: dataArg * 2;               / do the work
  h: .z.w;                           / the caller's handle
  (neg h) (callbackName; result);    / call back asynchronously
}

Two details make this work. Read .z.w inside the server function, where it names the calling connection; and make the callback asynchronous (neg h), so that neither side blocks.

The examples below use 0N! to print to the console of whichever process runs it.

A single argument and a callback

A client calls a remote function with one argument and names a callback to receive the result.

On the server, define proc to take a data argument x and a callback name y. It does its work, then uses the caller's handle to invoke the callback:

\p 5000                                        / listen on port 5000
serverFunc: {0N!x;}                            / stands in for the real server logic
proc: {serverFunc x; h: .z.w; (neg h) (y; 43)} / the function the client calls

On the client, define the callback, open a handle, and call proc asynchronously:

clientFunc: {0N!x;}   / receives the result
h: hopen `::5000
(neg h) (`proc; 42; `clientFunc)

The server console shows 42, and the client console then shows 43.

Several arguments

To call a remote function with more than one data argument, pass them as a list.

On the server, define proc3 to take a list of arguments x and a callback name y. It uses Apply (.) to spread the list across the parameters of add3:

\p 5000
add3: {x+y+z}                                  / takes three arguments
proc3: {r: add3 . x; 0N!r; (neg .z.w) (y; r)}  / unpacks the list, then calls back

On the client, send the arguments as a single list:

clientFunc: {0N!x;}
h: hopen `::5000
(neg h) (`proc3; 1 2 3; `clientFunc)

The server console shows 6, and the client console then shows 6.

A generic dispatcher

Rather than write a wrapper for every server-side function, define one generic wrapper, dispatcher. A client can then call any server-side function without new code on the server.

On the server, dispatcher takes the target function's name, its arguments as a list, and the client's callback name. It resolves the name, applies the arguments, and calls back with the result:

\p 5000
add3: {x+y+z}                                 / an arbitrary server function
dispatcher: {(neg .z.w) (z; (value x) . y)}   / runs any function of any rank

On the client, name the target function, its arguments, and the callback:

clientFunc: {0N!x;}
h: hopen `::5000
(neg h) (`dispatcher; `add3; 1 2 3; `clientFunc)

The client console shows 6. The server does the arithmetic but prints nothing.

An anonymous function

Send a function literal from the client to run on the server. This needs no predefined function on the server at all.

On the server, start a q process listening on a port, with no definitions:

q -p 5000

On the client, send an anonymous function that carries both the work and the callback:

q)clientFunc: {0N!x;}
q)h: hopen `::5000
q)(neg h) ({(neg .z.w) (z; x*y)}; 6; 7; `clientFunc)
42

The message carries the anonymous function {(neg .z.w) (z; x*y)}, its arguments 6 and 7, and the callback name `clientFunc. The server computes 6*7 and the client console shows 42.

An unprotected server runs whatever a client sends

Treat this pattern with great care outside a trusted network. A server that evaluates arbitrary client input is open to arbitrary code execution by any client that can connect. Restrict which functions a client can run before you expose such a server.

Blocking async

The point of an asynchronous message is that the sender does not wait. Sometimes, though, you want to send asynchronously and then block for a specific reply. To block until the next asynchronous message arrives on handle h, call h[].

This pattern is infrequent, and worth reaching for only in the two situations below: consuming an asynchronous interface as though it were synchronous, and starting remote work on several processes before you collect any of the results.

Also called deferred synchronous

Older material calls this pattern deferred synchronous, a name worth treating with care: the reply is not deferred, and nothing about the exchange is synchronous. Both messages are ordinary asynchronous messages; the only synchronous thing is that the sender chooses to block. Deferred response is a separate mechanism with a confusingly similar name — it suspends a genuinely synchronous reply with -30!.

Consume an asynchronous interface synchronously

Suppose a server offers a function you can't change, which returns its result through an asynchronous callback, and you need that result inside the function you are writing rather than in a callback that runs after your function has exited.

On the server:

\p 5000

/ Computes a result and sends it back to the calling client (.z.w)
/ asynchronously. Assume you cannot modify this function.
asyncAdd: {neg[.z.w] (`setAddResult; x+y)}

On the client:

q)h: hopen `::5000
q){
    neg[h](`asyncAdd; 40; 1);   / send the async request, expecting 41
    (`setAddResult; res): h[];  / block for the reply, whose shape you know
    res+1
    }[]
42

The assignment (`setAddResult; res): h[] is a list pattern: it checks that the reply starts with the expected symbol and binds the payload to res in one step, so an unexpected message raises an error instead of being silently misread.

Start remote work, then collect the results

To overlap remote work with local work, send every request first, flush each handle, do your local work, and only then block for the replies.

On both servers:

\p 5000   / and \p 5001 on the second process

/ Runs a query and sends the result back to the original client asynchronously
asyncWrap: {neg[.z.w] (`asyncWrapResult; value x)}
asyncAdd: {neg[.z.w] (`setAddResult; x+y)}

On the client:

q)(h1; h2): hopen each `::5000`::5001
q){
    neg[h1](`asyncWrap; (+; 10; 10));   / start remote work on the first server
    neg[h1][];                          / flush, or the message sits in the queue
    neg[h2](`asyncAdd; 10; 10);         / start remote work on the second server
    neg[h2][];

    two: 1+1;                           / local work, overlapping the remote work

    (`asyncWrapResult; res1): h1[];     / now collect the results
    (`setAddResult; res2): h2[];

    res1+res2+two
    }[]
42

Flushing matters here. Without neg[h1][] and neg[h2][], q would send the requests only when you reach h1[] and h2[], which defeats the purpose of starting them early.

Limits of blocking on a handle

h[] returns the next asynchronous message on the handle, whatever it is. That makes the pattern unsafe on a handle that also carries unsolicited asynchronous traffic.

On the server:

\p 5000
asyncAdd: {neg[.z.w] (`setAddResult; x+y)}

\t 10                                    / every 10 ms, send an unrelated message
.z.ts: {neg[key .z.W]@\: (abs; -42)}

On the client, the pattern works until an unrelated message arrives first:

q)while[1; neg[h](`asyncAdd; 10; 10); (`setAddResult; res): 0N!h[]]
(`setAddResult;20)
(`setAddResult;20)
(abs;-42)                                / not the message the pattern expects
'match
    [0]  while[1; neg[h](`asyncAdd; 10; 10); (`setAddResult; res): 0N!h[]]
                                              ^

Incoming synchronous queries don't interfere, because h[] serves them behind the scenes and keeps waiting for the first asynchronous message. Their side effects still reach your process, though, which can surprise code that doesn't expect them:

/ On the server: send a sync query with a side effect, frequently
\t 10
.z.ts: {key[.z.W]@\: (0N!; -42)}

/ On the client: the expected results still arrive, interleaved with the
/ output of the sync queries that h[] ran on your behalf
q)while[1; neg[h](`asyncAdd; 10; 10); (`setAddResult; res): 0N!h[]]
(`setAddResult;20)
-42                                      / printed by the sync query h[] ran
(`setAddResult;20)

Deadlocks

As already demonstrated, incoming sync requests are executed when the process is waiting for an async message by handle[]. Also, both incoming sync and async requests are executed when waiting for the result of a sync query!

On the server: 

q)add:{h:.z.w;
       neg[h]({-1"<- async msg"};(::));
       forty:h({-1"<- sync query";x};40);
       forty+x+y}

On the client: 

q){-1"-> querying h...";res:h (`add;1;1);-1"<- querying h returned";res}[]
-> querying h...
<- async msg
<- sync query
<- querying h returned
42

Therefore, it is not possible to deadlock with just two processes exchanging sync queries and async messages!

However, there is a 3rd IPC action that potentially blocks: the async flush. When flushing large amounts of outgoing data to a handle when the receiving party is not cooperating with the reception, the sender will block. With both parties doing the same, it's a deadlock.

Both processes: 

q)neg[h] til 1000000000;neg[h][] /deadlock

Interestingly, with q IPC between two processes, the cause of a deadlock is not the reception of a message nobody wants to send, but rather the sending of messages nobody wants to receive!

Note that it's flushing that causes the problem. Without flush, you can safely send any amount of data. (If the receiver isn't cooperating, the sender will eat all RAM, but no deadlock.)

Also note that when sending an sync query, the process must first actually send the query message to the peer, therefore every sync query inherently involves a flush behind the scenes. Therefore, two processes can deadlock if they mix sending large amounts of data and synchrous requests.

Both processes: 

q)neg[h]til 1000000000;h"1+2" /deadlock

Also, simply sending a large sync request can deadlock:

Both processes: 

q)h(neg;til 1000000000) /deadlock

So we can conclude that when only two processes communicate, deadlocks can only happen if there are large messages involved. However, when there are three or more processes, a deadlock is much easier to achieve. When multiple processes send syncronous queries or blocking async's (or a mixture of the two), a directed circle is a deadlock: e.g. P1 querying P2, P2 querying P3 and P3 querying P1 -- regardless of message sizes. To design a guaranteed deadlock-free group of interconnected processes, do not allow directed circles, i.e. make the communication pattern a DAG (Directed Acyclic Graph).

Next steps