Parallelization

Secondary q threads

PyKX starts embedded q with as many secondary q threads enabled as are available. These threads are automatically used by q to parallelize some computations as it deems appropriate. The QARGS environment variable can be used to provide command-line arguments and other startup flags to q/PyKX, including the number of secondary threads:

QARGS='-s 0' python # disable secondary threads

QARGS='-s 12' python # use 12 secondary threads by default

The value set using -s provides both the default, and the maximum available to the process - it cannot be changed after PyKX has been imported.

PyKX exposes this maximum value as pykx.Q.max_num_threads, which cannot be assigned to. The current number of secondary threads being used by q is exposed as pykx.Q.num_threads. It is initially equal to pykx.Q.max_num_threads, but can be assigned to a lower value.

Multi-threading

PyKX does not currently support calling into q from multiple threads within a Python process simultaneously. The GIL generally prevents this from occuring.

Peach

Having q use peach to call into Python is not supported, and will hang indefinitely.

For example, calling from Python into q into Python works normally:

>>> kx.q('{x each 1 2 3}', lambda x: range(x))
pykx.List(pykx.q('
,0
0 1
0 1 2
'))

But calling from Python into q into Python using peach hangs:

>>> kx.q('{x peach 1 2 3}', lambda x: range(x)) # Warning: will hang indefinitely