Skip to content

Handling concurrent queries

This page explains how to serve many concurrent users from one database: the three ways to spread queries across a pool of HDB processes, what each costs, and why the most capable of them cannot be built in q.

A q process serves one query at a time. One expensive query delays every other client connected to that process, and no amount of tuning changes that — you solve concurrency for a historical database (HDB) by running more than one HDB.

Every approach below therefore has the same shape: run several HDB processes against the same database, and put something in front of them to spread the work. Running several is safe and cheap. Mapped data is read-only and shared by the operating system. As a result, nothing is locked and nothing is copied. The processes even share the benefit of a warmed page cache. What differs is what does the spreading.

TCP socket sharding: the cheapest option

Built into q, and the cheapest thing that works. Start each HDB with the rp prefix on the same port, and the kernel distributes incoming connections across them:

q hdb.q -p rp,5010    # on one host, started several times

Clients connect to a single address, with no extra component to build or run, and no change to client code.

These limits matter, though: the kernel balances connections, not query cost, and knows nothing about which process is busy — it might route a new connection to a listener that is blocked mid-query, leaving that client waiting while another process sits idle. It also cannot route by content, aggregate across processes, or fail over.

See load balancing for how to enable it, and a load-balancing KDB-X server for a worked implementation.

A kdb+ gateway: the standard option

A gateway is a q process that clients connect to instead of the HDBs. It holds connections to the back-end processes, tracks which are free, forwards each query to one of them, and returns the result.

Because the gateway is itself q, it can do far more than balance load:

  • Route by content — today's data to the real-time database (RDB), historical data to an HDB
  • Split one client query across several processes and aggregate the results
  • Cache results
  • Enforce entitlements
  • Handle a back end dying mid-query

This flexibility is why it's the standard setup in production systems. The cost comes from a component that you must design, write, and maintain, and from a hop that every query pays.

A gateway must never block

A gateway is itself a q process, so it too serves one message at a time. If it sends a query to an HDB and waits synchronously for the answer, it is blocked for the duration and every other client waits behind it — the gateway becomes the bottleneck it was built to remove.

Gateways therefore talk to their back ends asynchronously, and use deferred response to hold a client's request open while the work happens elsewhere and reply later. Getting this right is the central problem of gateway design.

See gateway design for the design principles, query routing for a full worked framework, and a load-balancing server for a minimal implementation.

A protocol-aware load balancer: the fastest option

A dedicated load balancer that understands the kdb+ interprocess communication (IPC) protocol sits in front of the pool and is transparent to both sides: clients connect to it as though it were an HDB, and the HDBs need no gateway logic of their own.

Because it parses the protocol rather than merely accepting TCP connections, such a balancer can dispatch per query rather than per connection, health-check back ends properly, drain a process for maintenance, and absorb a back-end restart without breaking client connections. It is the option to reach for when the pool is large or availability requirements are strict.

Why you cannot write this one in q

This tier is not merely a gateway written better. A transparent load balancer has to move data between clients and servers in a way q cannot express, which is why it belongs outside q rather than in it.

A q process handling a message fully receives and deserializes it before it can act on it, in both directions. q materializes every message in memory as q objects, then serializes it again onto an output buffer to forward it. Three consequences follow:

  • Memory use exceeds the data in flight. The deserialized objects plus the outgoing buffer are larger than the message itself. With several clients and several servers sending in parallel, many such messages sit in input and output buffers at once. As a result, consumption swings widely with the traffic mix rather than staying flat. Under real load a q load balancer hits its memory limit at moments you cannot predict.
  • The symbol pool grows and never shrinks. Deserializing a message interns every symbol in it into the process's global symbol pool, and that pool is never reclaimed for the life of the process. A component whose whole job is to pass other people's data through therefore accumulates every symbol it has ever forwarded.
  • Every hop adds latency. Deserializing and re-serializing is work that contributes nothing to the answer, and nothing can be forwarded until the whole message has arrived.

Think of it as a railway. A q load balancer brings each train to a full stop at the station, unloads all its cargo into temporary storage, and only once unloading is complete loads it onto the outbound train — one train at a time.

An efficient load balancer never fully receives a message. It streams the bytes through, holding only a sliding window of any message in memory at a time, so consumption is constant regardless of message size. It does not deserialize, so nothing enters a symbol pool. The trains do not stop, and many pass through in parallel.

That streaming model is what q has no way to express, and it is the reason this tier exists as a separate component rather than as a better-written gateway.

Denial-of-service protection

You have to defend a shared pool against any one of its users. A poorly written client — or several eager ones — can fire many queries at once, occupying every process in the pool and starving everybody else. Whether that is accidental or deliberate, the effect is the same outage.

The obvious defense is a quota: cap how many queries one user may run concurrently. It works, but it does not scale as an operational practice, because a single uniform quota cannot be right for everyone:

  • Set it high enough for the heaviest legitimate user, and it is too high to prevent starvation.
  • Set it low enough to guarantee fairness, and most of the pool sits idle most of the time.

The usual escape is per-user quotas, continuously retuned as usage changes — which trades the outage for a permanent operational burden, or for a pool sized well beyond what the workload needs. Neither is free: one costs an operations team's attention, the other costs hardware.

A load balancer whose dispatch logic accounts for what each back end is already doing avoids the dilemma: it can keep utilization high and still refuse to let one client monopolize the pool, without any per-user configuration to maintain. That is a stronger argument for the dedicated component than raw throughput.

Next steps