Skip to content

How to tune Linux and the BIOS for KDB-X

This page explains the Linux and BIOS settings that can affect a KDB-X host. Measure the defaults first, change one setting at a time, and retain a change only when the representative workload improves.

Operating-system tuning cannot compensate for a query that reads the wrong partitions or columns. Start with the data layout and bottleneck evidence. Kernel, firmware, distribution, storage-driver, and cloud-platform versions can change the best setting, so avoid applying an old tuning checklist unchanged.

Record the host state with tools such as uname -a, lscpu, numactl --hardware, lsblk, findmnt, and sysctl. Keep the output with the benchmark so that a result can be reproduced after an upgrade.

NUMA

On a multi-socket or multi-node host, KDB-X is not NUMA-aware. Linux decides where the process runs and where its pages are allocated. Remote memory has higher latency and can make otherwise identical runs vary.

First inspect the topology and the placement of the running process:

numactl --hardware
numastat -p PID

Where the working set fits on one NUMA node, keep its CPUs and memory local. A preferred policy can fall back to another node if local memory is exhausted; a strict bind cannot:

numactl --cpunodebind=0 --preferred=0 q hdb.q -s 8
numactl --cpunodebind=0 --membind=0 q hdb.q -s 8    # strict; size first

On a larger host, running one q process per NUMA node is often easier to predict than making one process span every socket. If one process must span nodes, benchmark an interleaved policy against the default:

numactl --interleave=all q hdb.q -s 32

Do not set vm.zone_reclaim_mode or interleave memory from a blanket recommendation. Zone reclaim trades local allocation against reclaim work and page-cache retention; the result depends on the kernel and whether the process is confined to a node. KX recommendations have also changed across kernel generations. Follow the KX Linux production notes for the deployed kernel, then verify with numastat, page-fault, swap, and query-latency measurements.

File system

The same device can produce different KDB-X results under different file systems. KX's nano file-system tests found large differences in concurrent writes, sorting, and applying attributes, while no file system won every read pattern. Treat those results as a shortlist, not a universal ranking.

Common Linux choices include:

  • ext4 and XFS, mature journaling file systems supported by the major distributions;
  • Btrfs and ZFS, copy-on-write file systems with checksums, snapshots, and optional transparent compression; and
  • F2FS, designed for flash storage.

Choose with the complete workload:

  • Ingestion and end-of-day writes can differ much more than cached reads. Include concurrent writers, sorting, attributes, and compression in the test.
  • Cold and warm reads exercise different paths. Run both, with the actual partition and query sizes.
  • Copy-on-write behavior affects file rewrites and snapshots. Snapshot convenience can be worth a write cost, but measure it.
  • Operating-system support determines which file-system versions and repair tools are supportable in production.
  • File-system compression is optional, including on ZFS, Btrfs, and F2FS. Do not assume it is enabled. Avoid compressing the same data again with KDB-X file compression unless a benchmark shows a benefit.

Different storage tiers can use different file systems. Test each tier with its own read, write, retention, and recovery pattern before formatting it.

Combine disks

A single device can limit a query that scans columns. Several devices can be combined below KDB-X:

  • A striped LVM logical volume distributes fixed-size stripes across physical volumes.
  • RAID 0 stripes for performance without redundancy. The failure of one member loses the volume.
  • RAID 1 or RAID 10 mirrors data, trading capacity for redundancy and read options.
  • RAID 5 or RAID 6 stores parity, preserving more capacity but adding write and rebuild work.

Compare this with a segmented database:

  • Striping spreads every file across devices, so even one partition can use several devices. KDB-X sees one directory.
  • Segmentation places whole partitions on separate file systems. A query uses several devices only when it spans partitions on those segments, but tiers and failure domains can be managed separately.

A segmented database can also use striped volumes as its segments. Benchmark single-partition and multi-partition queries, ingestion, degraded RAID operation, and rebuilds. The stripe size, RAID level, controller cache, and file-system alignment can all change the result.

Read-ahead

Linux read-ahead fetches blocks that follow a detected sequential read. A larger value can help scans on HDD or high-latency storage. It can waste I/O and page cache for selective or random access.

Show and set block-device read-ahead in 512-byte sectors:

sudo blockdev --getra /dev/nvme0n1
sudo blockdev --setra 4096 /dev/nvme0n1   # 2 MiB

Test cold reads at several values; warm reads do not measure the device setting. With device mapper, LVM, RAID, or network storage, confirm which logical and underlying devices have effective read-ahead. Persist a successful value with the distribution's device or startup configuration rather than an ad hoc command.

Memory pages

Linux commonly uses 4 KiB base pages on x86-64. Transparent Huge Pages (THP) can reduce address-translation work for a large heap, but promotion, compaction, splitting, and kernel-specific defects can introduce pauses. KX guidance differs by kernel generation, so neither "always disable THP" nor "always enable THP" is a safe general rule.

Inspect both the use and defragmentation policy:

cat /sys/kernel/mm/transparent_hugepage/enabled
cat /sys/kernel/mm/transparent_hugepage/defrag

Start with the supported distribution and KX recommendation for the deployed kernel. If a process shows unexplained latency spikes or CPU spent in memory compaction, compare policies on an isolated host with the full workload and memory pressure. Restart q between tests where the setting requires it.

Simultaneous multithreading

Simultaneous multithreading (SMT), called Hyper-Threading on Intel processors, exposes two hardware threads per physical core. They share core execution resources and caches.

SMT can improve throughput when threads often wait for memory or storage. It can hurt a compute-bound workload or a latency-sensitive main thread that shares a physical core. Size secondary threads initially from .z.c, which reports physical cores, then benchmark the workload with SMT enabled and disabled. Pin the main thread or latency-sensitive process away from competing work when isolation matters.

CPU power and affinity

Frequency scaling and deep idle states save power but can add wake-up latency. For a latency-sensitive process, compare the platform's performance profile with its default balanced profile:

cpupower frequency-info
sudo cpupower frequency-set -g performance

The available controls differ with the CPU driver, firmware, hypervisor, and cloud platform. A maximum-performance policy increases energy use and heat and can reduce sustained turbo frequency if cooling is insufficient. Measure a long run, not only the first seconds.

Use taskset, numactl --physcpubind, cpusets, or orchestrator CPU reservations to isolate important processes. Keep CPU affinity consistent with NUMA memory placement. Do not pin more busy q threads to a CPU set than it can run.

Resource limits and swap

A partitioned database can touch many files and mappings. Check the limits before a broad query or .Q.MAP:

ulimit -n
sysctl vm.max_map_count vm.swappiness
  • Raise the open-file limit for the service account when the measured workload needs more descriptors. .Q.MAP retains handles, and compressed files can require additional descriptors.
  • Raise vm.max_map_count when the process maps more areas than the default permits. Derive the requirement from partitions, tables, columns, compression, and concurrency, and leave headroom.
  • Size RAM so the q heaps and essential page cache do not swap under normal load. Lowering vm.swappiness can make anonymous-memory swapping less eager, but it does not create memory or guarantee that swap is never used.
  • Use a workspace limit and cgroup or container memory limit deliberately. Leave room outside the q heap for mapped resident pages, libraries, the symbol table, the operating system, and service sidecars.

Disabling swap entirely changes an overload from severe latency to an earlier out-of-memory kill. Choose that failure policy explicitly and test recovery instead of treating it as a performance toggle.

Network

Linux autotunes TCP buffers within configured limits. Increase the maximum receive or send buffer only when measurement shows that a connection cannot fill the bandwidth-delay product and the application is socket-buffer limited.

Use ss -ti to inspect congestion, retransmission, and window behavior, and a tool such as iperf3 to separate the network path from q serialization. For remote storage, test the storage protocol and query workload as well: a network throughput test does not include metadata requests or storage-controller limits.

At high packet rates, receive-side scaling, interrupt affinity, and NIC queues can compete with latency-sensitive q cores. Inspect per-CPU interrupt and softirq load before changing them, and keep any IRQ affinity consistent with NUMA placement.

Synchronize clocks with the platform's supported NTP or PTP service when latency is measured across hosts. Otherwise, one-way timings can report clock error rather than application latency.

Next steps