Skip to content

Handling concurrent queries

This page explains how to serve many concurrent users from one database: the three ways to spread queries across a pool of HDB processes, what each costs, and why the most capable of them cannot be built in q.

A q process serves one query at a time. One expensive query delays every other client connected to that process, and no amount of tuning changes that — concurrency for an HDB is solved by running more than one HDB.

Every approach below therefore has the same shape: run several HDB processes against the same database, and put something in front of them to spread the work. Running several is safe and cheap — mapped data is read-only and shared by the operating system, so nothing is locked and nothing is copied, and they even share the benefit of a warmed page cache. What differs is what does the spreading.

TCP socket sharding: a basic load balancer

Built into q, and the cheapest thing that works. Start each HDB with the rp prefix on the same port, and the kernel distributes incoming connections across them:

q hdb.q -p rp,5010    # on one host, started several times

Clients connect to a single address, with no extra component to build or run, and no change to client code.

These limits matter though: the kernel balances connections, not query cost, and knows nothing about which process is busy — it may route a new connection to a listener that is blocked mid-query, leaving that client waiting while another process sits idle. It also cannot route by content, aggregate across processes, or fail over.

See socket sharding for supported patterns, including dynamic scaling and rolling updates, and for a worked demonstration of the busy-listener behavior.

A kdb+ gateway: the widespread solution

A gateway is a q process that clients connect to instead of the HDBs. It holds connections to the back-end processes, tracks which are free, forwards each query to one of them, and returns the result.

Because the gateway is itself q, it can do far more than balance load. It can route by content — today's data to the RDB, historical data to an HDB — and split one client query across several processes and aggregate the results, cache, enforce entitlements, and handle a back-end dying mid-query. This flexibility is why it's the standard setup in production systems. The cost comes from a component that you must design, write, and maintain, and from a hop that every query pays.

A gateway must never block

A gateway is itself a q process, so it too serves one message at a time. If it sends a query to an HDB and waits synchronously for the answer, it is blocked for the duration and every other client waits behind it — the gateway becomes the bottleneck it was built to remove.

Gateways therefore talk to their back ends asynchronously, or use deferred response to hold a client's request open while the work happens elsewhere and reply later. Getting this right is the central problem of gateway design.

See gateway design for the design principles, query routing for a full worked framework, and a load-balancing server for a minimal implementation.

A protocol-aware load balancer: the performant solution

A dedicated load balancer that understands the kdb+ IPC protocol sits in front of the pool and is transparent to both sides: clients connect to it as though it were an HDB, and the HDBs need no gateway logic of their own.

Because it parses the protocol rather than merely accepting TCP connections, such a balancer can typically dispatch per query rather than per connection, health-check back ends properly, drain a process for maintenance, and absorb a back-end restart without breaking client connections. It is the option to reach for when the pool is large or availability requirements are strict.

Why this one cannot be written in q

This tier is not simply "a gateway written better". A transparent load balancer has to move data between clients and servers in a way q cannot express, which is why it belongs outside q rather than in it.

A q process handling a message fully receives and deserializes it before it can act on it, in both directions. Every message is materialized in memory as q objects, then serialized again onto an output buffer to be forwarded. Three consequences follow:

  • Memory use exceeds the data in flight. The deserialized objects plus the outgoing buffer are larger than the message itself. With several clients and several servers sending in parallel, many such messages sit in input and output buffers at once, so consumption swings widely with the traffic mix rather than staying flat. Under real load a q load balancer will hit its memory limit at moments you cannot predict.
  • The symbol pool grows and never shrinks. Deserializing a message interns every symbol in it into the process's global symbol pool, and that pool is never reclaimed for the life of the process. A component whose whole job is to pass other people's data through therefore accumulates every symbol it has ever forwarded.
  • Latency is added at every hop. Deserializing and re-serializing is work that contributes nothing to the answer, and nothing can be forwarded until the whole message has arrived.

Think of it as a railway. A q load balancer brings each train to a full stop at the station, unloads all its cargo into temporary storage, and only once unloading is complete loads it onto the outbound train — one train at a time.

An efficient load balancer never fully receives a message. It streams the bytes through, holding only a sliding window of any message in memory at a time, so consumption is constant regardless of message size. It does not deserialize, so nothing enters a symbol pool. The trains do not stop, and many pass through in parallel.

That streaming model is what q has no way to express, and it is the reason this tier exists as a separate component rather than as a better-written gateway.

Denial-of-service protection

A pool shared between users needs to be defended from any one of them. A poorly written client — or simply several eager ones — can fire many queries at once, occupying every process in the pool and starving everybody else. Whether that is accidental or deliberate, the effect is the same outage.

The obvious defence is a quota: cap how many queries one user may run concurrently. It works, but it does not scale as an operational practice, because a single uniform quota cannot be right for everyone:

  • Set it high enough for the heaviest legitimate user, and it is too high to prevent starvation.
  • Set it low enough to guarantee fairness, and most of the pool sits idle most of the time.

The usual escape is per-user quotas, continuously retuned as usage changes — which trades the outage for a permanent operational burden, or for a pool sized well beyond what the workload needs. Neither is free: one costs an operations team's attention, the other costs hardware.

A load balancer whose dispatch logic accounts for what each back end is already doing avoids the dilemma: it can keep utilization high and still refuse to let one client monopolize the pool, without any per-user configuration to maintain. That is a stronger argument for the dedicated component than raw throughput.

Next steps