Internal functions

The operator ! with a negative integer as left argument calls an internal function.


Internal functions are for use by language implementors. They may be redefined in subsequent releases.

They also allow new language features to be tried on a provisional basis. Where they are replaced by keywords or utilities, use the replacements.

Exposed infrastructure

-4!x        tokens                  Replaced:
-8!x        to bytes                 -1!x   hsym
-9!x        from bytes               -2!x   attr
-10!x       type enum                -3!x   .Q.s1
-11!x       streaming execute        -5!x   parse
−11!(-1;x)  streaming execute        -6!x   eval
−11!(n;x)   streaming execute        -7!x   hcount
-11!(-2;x)  logfile chunks           -12!x
-14!x       quote escape             -13!x  .Q.addr
-16!x       ref count                -15!x  md5
-17!x       flip endian-ess          -20!0  .Q.gc
-18!x       compress byte            -24!x  reval
-19!x       compress file            -29!x  .j.k
-21!x       compression stats       
-22!x       uncompressed length     
-23!x       memory map              
-25!x       async broadcast         
-26!x       SSL                     
-27!(x;y)   format                  
-30!x       deferred response       

Neal Stephenson thinks it’s cute to name his labels 'dengo'

-1!x (hsym)

Use hsym.

-2!x (attr)

Use attr.

-3!x (string)

Use .Q.s1.

show, string, .Q.s

-4!x (tokens)

Returns the list of q tokens found in string x. (Note the q parsing of names with embedded underscores.)

q)-4!"select this from that"
," "
," "
," "

q)-5!"select this from that" / compare with −5!

q)-4!"a variable named aa_bb"
," "
," "
," "

-5!x (parse)

Use parse.

-6!x (eval)

Use eval.

-7!x (hcount)


-8!x (to bytes)

Returns the IPC byte representation of x.

q)-8!1 2 3

-9!x (from bytes)

Creates data from IPC byte representation x.

q)-9!-8!1 2 3
1 2 3

-10!x (type enum)

Resolve a type number to an enum vector and check if it is available.

q)type vv

-11!x (streaming execute)

Streaming-execute over file x, used for example in kdb+tick to replay logfiles in a memory-efficient manner.

A logfile is just a list of lists, and each list is read in turn and evaluated, either by value or by if it is defined.

Here, for demonstration purposes, we manually create a logfile, and play it back through -11!. This is functionally equivalent to doing value each get `:logfile but uses far less memory.

q)`:logfile.2013.12.03 set () / create a new,empty log file
q)h:hopen `:logfile.2013.12.03 / open it
q)h enlist(`f;`a;10) / append a record
q)h enlist(`f;`b;20) / append a record
q)hclose h / close the file
q)/Define the function that is referenced in those records
q)-11!`:logfile.2013.12.03 / playback the logfile
q)/This is the whole purpose of -11!x.
q)value each get `:logfile.2013.12.03
`a 10
`b 20

If successful, the number of chunks executed is returned. If the end of the file is corrupt a 'badtail error is signalled. In the event that the log file references an undefined function, the function name is signalled as an error. This can be confusing if the missing function name is upd, as it does not reflect the same situation as the license expiry 'upd error. e.g.

/ Continuing the above example
q)delete f from `.
q)/function f no longer defined, so it signals an error
'f for examples of usage

−11!(-1;x) (streaming execute)

Same as −11!x.

−11!(n;x) (streaming execute)

Streaming-execute the first n chunks of logfile x, return the number of chunks if successful.

It is possible to use the above to playback n records from record M onwards.

Firstly create a sample log file, which contains 1000 records as ((f;0);(f;1);(f;2);..;(f;999)).

q)`:log set();h:hopen`:log;i:0;do[1000;h enlist(`f;i);i+:1];hclose h;

Then define function f to just print its arg, skip the first M records. If is defined, -11! calls it for each record.

q)m:0;M:750;f:0N!;{m+:1;if[m>M;value x;];};-11!(M+5-1;`:log)

-11!(-2;x) (logfile chunks)

Given a valid logfile, returns the number of chunks.

Given an invalid logfile, returns the number of valid chunks and length of the valid part.

q)logfile:`:good.log / a non-corrupted logfile
q)logfile:`:broken.log / a manually corrupted logfile
q)/define a dummy upd file as components are of the form (`upd;data)
q)hcount logfile
q)/ 26 valid chunks until position 35634 (out of 39623)

-12!x (


-13!x (.Q.addr)

Use .Q.addr.

-14!x (quote escape)

Handles " escaping in strings: used to prepare data for CSV export.

-15!x (md5)

Use md5.

-16!x (ref count)

Returns the reference count for a variable.

q)a:b:c:d:e:1 2 3

-17!x (flip endian-ess)

Returns flip endian-ness of kdb+ datafile x, see notes in Changes in kdb+ V2.6

-18!x (compress byte)

Returns compressed IPC byte representation of x, see notes about network compression in Changes in V2.6

-19!x (compress file)

Syntax: -19! x

Where x is a list of 5 items:

  • source file: file symbol
  • target file: file symbol
  • logical block size: a power of 2 between 12 and 20 (pageSize or allocation granularity to 1MB – pageSize for AMD64 is 4kB, SPARC is 8kB. Windows seems to have a default allocation granularity of 64kB). When choosing the logical block size, consider the minimum of all the platforms that will access the files directly – otherwise you may encounter "disk compression - bad logicalBlockSize". Note that this argument affects both compression speed and compression ratio: larger blocks can be slower and better compressed.
  • compression algorithm: one of:
    • 0: none
    • 1: q IPC
    • 2: gzip
    • 3: snappy (since V3.4)
    • 4: lz4hc (since V3.6)
  • compression level: an integer
    • for gzip: between 0 and 9
    • for lz4hc: between 1 and 12 (int x taken as 12&x)
    • otherwise: 0

returns the target file as a file symbol.

q)`:test set asc 10000000?100; / create a test data file
q) / compress input file test, to output file ztest
q) / using a block size of 128kB (2 xexp 17), gzip level 6
q) / check the compressed data is the same as the uncompressed data

lz4 compression

Certain releases of lz4 do not function correctly within kdb+.

Notably, lz4-1.7.5 does not compress, and lz4-1.8.0 appears to hang the process.

Kdb+ requires at least lz4-r129. lz4-1.8.3 works. We recommend using the latest lz4 release available.

Knowledge Base: File compression

-20!0 (.Q.gc)

Use .Q.gc.

-21!x (compression stats)

Syntax: -21! x

Where x is a file symbol, returns a dictionary of compression statistics for it.

compressedLength  | 137349
uncompressedLength| 80000016
algorithm         | 2i
logicalBlockSize  | 17i
zipLevel          | 6i

Knowledge Base: File compression

-22!x (uncompressed length)

An optimized shortcut to obtain the length of uncompressed serialized x, i.e. count -8!x

q)v:til 100000
q)\t do[5000;-22!v]
q)\t do[5000;count -8!v]
q)(-22!v)=count -8!v

-23!x (memory map)

Since V3.1t 2013.03.04

Attempts to force the object x to be resident in memory by hinting to the OS and/or faulting the underlying memory pages. Useful for triggering sequential access to the storage backing x.

-24!x (read-only eval)

Use reval.

-25!x (async broadcast)

Since V3.4

Broadcast data as an async msg to specified handles. The advantage of using -25!(handles;msg) over neg[handles]@\:msg is that -25!msg will serialize msg just once – thereby reducing CPU and memory load.

Use as

q)-25!(handles; msg)

Handles should be a vector of positive int or longs.

msg will be serialized just once, to the lowest capability of the list of handles. I.e. if handles are connected to a mix of versions of kdb+, it will serialize limited to the types supported by the lowest version. If there is an error, no messages will have been sent, and it will return the handle whose cap caused the error.

Just as with neg[handles]@\:msg, -25!x queues the msg as async on those handles – they don't get sent until the next spin of the main loop, or are flushed with neg[handles]@\:(::).

-25!(handles; ::) can also flush the handles

Possible error scenarios:

  • from trying to serialize data for a handle whose remote end does not support a type, or size of the data.

    / connect to 2.8 and 3.4
    q)h:hopen each 5000 5001
    5 6i 
    q)(-5) 0Ng / 2.8 does not support guid
    q)(-6) 0Ng / 3.4 does support guid 
    'type error serializing for handle 5

  • an int is passed which is not a handle

    q)-25!(7 8;0Ng)
    '7 is not an ipc handle

-26!x (SSL)

View TLS settings on a handle or current process -26!handle or -26!(). Since V3.4 2016.05.12.

Knowledge Base: SSL

-27!(x;y) (format)


  • x is an int atom
  • y is numeric

returns y as a string or strings formatted as a float to x decimal places. (Since V3.6 2018.09.26.) It is atomic and doesn’t take \P into account. e.g.

q)-27!(3i;0 1+123456789.4567)

This is a more precise, built-in version of .Q.f but uses IEEE754 rounding:


You might want to apply a rounding before applying -27!.

-29!x (parse JSON)

Use .j.k.

-30!x (deferred response)

Syntax: -30!(::)
Syntax: -30!(handle;isError;msg)

Where handle is an int, isError is a boolean, and msg is a string

  • -30!(::) allows the currently-executing callback to complete without responding
  • -30!(handle;isError;msg) responds to the deferred sync call

Since V3.6 2018.05.18.

Knowledge Base: Deferred response