Send Feedback
Skip to content

How to Check Memory Usage

This guide walks through how to measure and understand memory usage in KDB-X using built-in tools and commands.

System tools

Each operating system offers tools to view memory usage at the process level:

  • Windows: Task Manager
  • macOS: Activity Monitor
  • Linux/Unix: top, htop, ps, etc.

For more detailed control within q, use the built-in functions covered below. Whether you're profiling performance or debugging memory pressure, these steps will help you monitor memory effectively.

Overview

Measure memory used by q expressions

Use the system command \ts to profile how much memory is used by a specific expression. In the example below,\ts times the creation of one million longs.

q)\ts til 1000000
4 8388784

The output shows that the operation took 4 microseconds and used approximately 8 MB of memory within the KDB-X memory pool.

Another example measures the time and memory used to convert the last element of a one-million-item list to a string and concatenate it with a text prefix.

q)a:til 1000000
q)\ts "Last number is ",string last a
0 1088

The result shows that the operation took 0 microseconds and used roughly 1 KB of memory.

Check memory usage of the q session

Use .Q.w[] or \w to retrieve session memory statistics.

Heap usage

KDB-X manages its own memory pool, referred to as the heap. This heap represents memory that the current process has allocated from the underlying operating system.

Heap statistics provide insight into how memory is used during a session and how it relates to the total memory allocated.

Key heap stats:

  • used – the amount of memory currently in use, allocated from the heap
  • heap – the total size of the heap, including both used and available memory
  • peak – the highest value heap has reached during the session
q).Q.w[]`used`heap`peak      / memory used prior to test
371584 67108864 67108864
q)til 1000000;               / temporarily create 1000000 longs
q).Q.w[]`used`heap`peak      / memory used remains the same
371584 67108864 67108864
q)til 10000000;              / temporarily create 10000000 longs
q).Q.w[]`used`heap`peak      / heap grows to create the 10000000 longs
371584 201326592 201326592
q)a:til 10000000             / 10000000 longs now assigned to a variable
q).Q.w[]`used`heap`peak      / memory used grows as there's now a variable referencing 10000000 longs
134589360 201326592 201326592

Per-thread heap

Each thread in q has its own heap. Use -s to start multiple threads.

q).Q.w[]`used`heap`peak
404624 67108864 67108864
q){show .Q.w[]`used`heap`peak} peach til 2
368 67108864 67108864
368 67108864 67108864

Memory limits

The wmax stat indicates the memory limit and defaults to 0 (unlimited).

q).Q.w[]`wmax
0

You can set this at start-up via the command line option -w, or at runtime via the system command \w.

q).Q.w[]`wmax
134217728
q)til 10000000
'-w abort

Understand memory mapping

Memory mapping allows KDB-X to access on-disk data as if it were in memory, enabling faster load times and efficient access to large datasets without reading entire files into memory.

  • Use the mmap stat from .Q.w[] to inspect the current amount of mapped memory.
  • Use get to load the data in immediate mode to instruct the file to be mapped into memory.
q)`:tab/ set ([]100000?100;100000?1000f)   / save table to 'tab'
`:tab/
q).Q.w[]`mmap                             / nothing memory mapped
0
q)t:get`:tab                              / load table (no trailing slash to indicate should be memory mapped)
q).Q.w[]`mmap                             / table loaded and mapped into memory
1600032

For deferred mapping (on-demand), use a trailing slash when loading the file.

More info: Memory Mapping in KDB-X.

Track symbol memory usage

Creating a new named variable, regardless of the data being stored, creates a new symbol.

q).Q.w[]`syms
689
q)price:44.5
q).Q.w[]`syms
690

Symbols are never removed from the symbol cache. Therefore, deleting the variable doesn't reduce the sym count.

q)delete price from `.
`.
q)price
'price
    [0]  price
                ^
q).Q.w[]`syms
690

Recreating a variable with the same name does not increase the symbol cache, since the symbol already exists:

q).Q.w[]`syms
690
q)price:44
q).Q.w[]`syms
690

Commonly used variable names, such as x, are often created temporarily during startup and do not increase the symbol count when reused.

q).Q.w[]`syms
689
q)x:22
q).Q.w[]`syms
689

Understand reference counting

Each object in q has a reference count. Use -16! to inspect it.

q)a:22
q)-16!a
1i
q)b:a                  / assign 'a' to 'b'
q)-16!a                / object referenced by 'a' now has 2 references to it
2i
q)-16!b                / object referenced by 'b' now has 2 references to it
2i
q)b:33                 / 'b' reassigned new value, no longer references 'a'
q)-16!b                / 'b' now using new object with only 1 reference to it
1i
q)-16!a                / object referenced by 'a' has decreased its reference count
1i

When an object no longer has any references, it is freed. The memory it used may either be returned to the operating system or added to an internal free list, managed by KDB-X.

You can delete a variable from its namespace using:

q)a:22                 / create variable 'a'
q)delete a from `.     / delete 'a' from root namespace

Reduce memory with Garbage Collection

Set garbage collection behavior via -g at startup or \g at runtime.

To trigger manual garbage collection, run .Q.gc[]:

q).Q.gc[]

Summary

In this guide, you learned how to:

  • Use system tools and q commands to inspect memory usage
  • Measure memory allocation of expressions using \ts
  • Track heap growth and per-thread memory behavior
  • Monitor and manage symbol memory
  • Understand and work with reference counts
  • Apply garbage collection to reclaim memory

Understanding how memory is used helps keep your applications efficient and responsive, especially under load.

Next steps

Feel free to revisit this guide as you monitor and optimize memory usage in your KDB-X environment. Happy tuning!