Handling concurrent queries¶
This page explains how to serve many concurrent users from one database: the three ways to spread queries across a pool of HDB processes, what each costs, why the most capable of them cannot be built in q, and why serving queries on many threads of one process is rarely a better choice.
A q process serves one query at a time. One expensive query delays every other client connected to that process, and no amount of tuning changes that — you solve concurrency for a historical database (HDB) by running more than one HDB.
Every approach below therefore has the same shape: run several HDB processes against the same database, and put something in front of them to spread the work. Running several is safe and cheap. Mapped data is read-only and shared by the operating system. As a result, nothing is locked and nothing is copied. The processes even share the benefit of a warmed page cache. What differs is what does the spreading.
TCP socket sharding: the cheapest option¶
Built into q, and the cheapest thing that works. Start each HDB with the rp prefix on the same port, and the kernel distributes incoming connections across them:
q hdb.q -p rp,5010 # on one host, started several times
Clients connect to a single address, with no extra component to build or run, and no change to client code.
These limits matter, though: the kernel balances connections, not query cost, and knows nothing about which process is busy — it might route a new connection to a listener that is blocked mid-query, leaving that client waiting while another process sits idle. It also cannot route by content, aggregate across processes, or fail over.
See load balancing for how to enable it, and a load-balancing KDB-X server for a worked implementation.
A kdb+ gateway: the standard option¶
A gateway is a q process that clients connect to instead of the HDBs. It holds connections to the back-end processes, tracks which are free, forwards each query to one of them, and returns the result.
Because the gateway is itself q, it can do far more than balance load:
- Route by content — today's data to the real-time database (RDB), historical data to an HDB
- Split one client query across several processes and aggregate the results
- Cache results
- Enforce entitlements
- Handle a back end dying mid-query
This flexibility is why it's the standard setup in production systems. The cost comes from a component that you must design, write, and maintain, and from a hop that every query pays.
A gateway must never block
A gateway is itself a q process, so it too serves one message at a time. If it sends a query to an HDB and waits synchronously for the answer, it is blocked for the duration and every other client waits behind it — the gateway becomes the bottleneck it was built to remove.
Gateways therefore talk to their back ends asynchronously, and use deferred response to hold a client's request open while the work happens elsewhere and reply later. Getting this right is the central problem of gateway design.
See gateway design for the design principles, query routing for a full worked framework, and a load-balancing server for a minimal implementation.
A protocol-aware load balancer: the fastest option¶
A dedicated load balancer that understands the kdb+ interprocess communication (IPC) protocol sits in front of the pool and is transparent to both sides: clients connect to it as though it were an HDB, and the HDBs need no gateway logic of their own.
Because it parses the protocol rather than merely accepting TCP connections, such a balancer can dispatch per query rather than per connection, health-check back ends properly, drain a process for maintenance, and absorb a back-end restart without breaking client connections. It is the option to reach for when the pool is large or availability requirements are strict.
Why you cannot write this one in q¶
This tier is not merely a gateway written better. A transparent load balancer has to move data between clients and servers in a way q cannot express, which is why it belongs outside q rather than in it.
A q process handling a message fully receives and deserializes it before it can act on it, in both directions. q materializes every message in memory as q objects, then serializes it again onto an output buffer to forward it. Three consequences follow:
- Memory use exceeds the data in flight. The deserialized objects plus the outgoing buffer are larger than the message itself. With several clients and several servers sending in parallel, many such messages sit in input and output buffers at once. As a result, consumption swings widely with the traffic mix rather than staying flat. Under real load a q load balancer hits its memory limit at moments you cannot predict.
- The symbol pool grows and never shrinks. Deserializing a message interns every symbol in it into the process's global symbol pool, and that pool is never reclaimed for the life of the process. A component whose whole job is to pass other people's data through therefore accumulates every symbol it has ever forwarded.
- Every hop adds latency. Deserializing and re-serializing is work that contributes nothing to the answer, and nothing can be forwarded until the whole message has arrived.
Think of it as a railway. A q load balancer brings each train to a full stop at the station, unloads all its cargo into temporary storage, and only once unloading is complete loads it onto the outbound train — one train at a time.
An efficient load balancer never fully receives a message. It streams the bytes through, holding only a sliding window of any message in memory at a time, so consumption is constant regardless of message size. It does not deserialize, so nothing enters a symbol pool. The trains do not stop, and many pass through in parallel.
That streaming model is what q has no way to express, and it is the reason this tier exists as a separate component rather than as a better-written gateway.
Denial-of-service protection¶
You have to defend a shared pool against any one of its users. A poorly written client — or several eager ones — can fire many queries at once, occupying every process in the pool and starving everybody else. Whether that is accidental or deliberate, the effect is the same outage.
The obvious defense is a quota: cap how many queries one user may run concurrently. It works, but it does not scale as an operational practice, because a single uniform quota cannot be right for everyone:
- Set it high enough for the heaviest legitimate user, and it is too high to prevent starvation.
- Set it low enough to guarantee fairness, and most of the pool sits idle most of the time.
The usual escape is per-user quotas, continuously retuned as usage changes — which trades the outage for a permanent operational burden, or for a pool sized well beyond what the workload needs. Neither is free: one costs an operations team's attention, the other costs hardware.
A load balancer whose dispatch logic accounts for what each back end is already doing avoids the dilemma: it can keep utilization high and still refuse to let one client monopolize the pool, without any per-user configuration to maintain. That is a stronger argument for the dedicated component than raw throughput.
Multithreaded input mode: one process, many threads¶
A single HDB can serve queries concurrently. Put a minus sign before the port number when you start it, and q runs every incoming connection on a thread of its own:
q hdb.q -p -5010
This is tempting for an HDB. The data is read-only, the queries are read-only, and there is no pool, gateway or load balancer to run. The main thread can still update globals, from .z.ts or from a socket it opened itself, so it can reload the database on a timer or when a subscription tells it to. On uncompressed data concurrent queries do overlap: in one test, 8 clients finished in 0.8 seconds in multithreaded mode, against 1.9 seconds when the same process ran them one at a time.
Its limitations still make a pool of HDBs behind a load balancer the better choice for most deployments. In order of importance:
- One query that needs too much memory kills every query. The
-wlimit applies to the process, and the memory of all connection threads counts towards it. When a query pushes the total past the limit, the whole process aborts with-w abort. Every client loses its connection and every running query is lost with it, even if the culprit needs less than the limit on its own and was simply unlucky in what ran beside it. In a pool, each HDB runs one query at a time, so the only query that fails is the one that caused the problem. The other HDBs keep serving. - Use
-g 1. Each connection thread has its own heap, and in the default deferred mode a thread keeps the memory of its largest query until the client disconnects. Long-lived client connections are normal, so idle connections hold on to their peaks, and those peaks count towards-w..Q.gc[]cannot fix this from outside: called from the main thread, it reclaims only the main thread's heap. Immediate mode returns each query's large blocks as soon as they are freed. That costs almost nothing, because handing a block back is a cheapmunmap: query times with-g 1were within measurement noise of-g 0, including queries that returned millions of rows. - You cannot limit concurrency. Each new connection gets a thread and starts running queries at once, up to a hard maximum of 1020 connections, and each thread needs at least 64 MB of heap plus the working space of its query. The process cannot queue requests: it has no way to hold the N+1th request until a running query finishes.
.z.poand.z.pcare not called in this mode..z.pwruns on the connection's own thread, so it cannot update a global counter, and.z.Wlists only the main thread's sockets. Any cap has to come from outside: a fixed pool of client connections, or a gateway. A pool caps concurrency by its size and queues the requests beyond it. - Use it only with uncompressed data. Reading a compressed file maps, decompresses and unmaps it block by block. Each of those system calls takes a lock that is shared by the whole process, so the connection threads queue behind each other on it. Separate HDB processes each have their own lock, so a pool does not suffer from this.
- A reload stops every query. The main thread takes a multiple-read, single-write lock to update globals. A reload waits for the running queries to finish and holds back new ones until it completes, so every client pauses together. A pool can reload one HDB at a time while the others keep serving.
- Loading the database can be slower. Secondary threads cannot change global variables, so the load cannot fill cached counts such as
.Q.pnlazily. Outside this mode,.Q.pnstarts empty and fills as queries touch partitions. In this mode, the load populates it up front as a workaround. With many partitions and tables, this can take a few seconds, because q must read the header of every (partition, table) pair. - There is no query timeout.
-Tcannot be combined with multithreaded input mode. The process refuses to start withcan't enable timeout with -port active. A runaway query occupies its thread, and its memory, until it finishes or the process aborts. On a pool,-Tfrees the HDB for the next query.
Multithreaded input mode fits a narrow case: an uncompressed database, a known and small number of clients, queries whose memory needs you control, and infrequent reloads. Outside that case, run several HDBs and spread the load across them.
Next steps¶
- Understand why one process serves one query at a time in the HDB process.
- Warm the page cache once for the whole pool with pre-fetching.
- Read gateway design and query routing before writing a gateway.
- See load balancing for the kernel-level option in detail.