Skip to content

Modes of operation

This page explains how to operate PyKX in Python and q, with or without a KDB Insights license.

PyKX can operate in different modes. Each mode has its limitations and requirements, so we're breaking them down into the following:

  1. PyKX within Python
    • 1.a Unlicensed
    • 1.b Licensed
  2. PyKX within q with a kdb Insights license

1. PyKX within Python

The purpose of operating PyKX within a Python session is to replace qPython and PyQ. Within Python, PyKX has two modes of operation:

  • Licensed (this means you have a kdb Insights license with PyKX enabled)
  • Unlicensed (this means you don't have a kdb Insights license or a license in which PyKX is not enabled)

The main difference between the two is that the Unlicensed mode is for IPC-based communication. This mean that it allows to create IPC connections and convert data from Pythonic representations to PyKX objects.

The following table outlines more key differences:

Feature Licensed Unlicensed
Convert objects from q to Pythonic types and vice-versa
Query synchronously and asynchronously a q server via IPC
Query synchronously and asynchronously a q server with TLS enabled
Interact with PyKX tables via a Pandas like API
Run arbitrary q code within a Python session
Display PyKX/q objects within a Python session
Load kdb+ Databases within a Python session
Read/write JSON, CSV and q formats to/from disk
Access to Python classes for SQL, schema creation, custom data conversion
Run Python within a q session using PyKX under q
Full support for nulls, infinities, data slicing and casting
Production support

1.a Running in Unlicensed mode

Unlicensed mode is a feature-limited mode of operation for PyKX. Its aim is to replace qPython, which has the benefit of not requiring a valid q license (except for the q license required to run the remote q process that PyKX connects to in this mode).

This mode cannot run q embedded within it. Also, it lacks the ability to run q code within the local Python process or any functionality that depends on running q code. Despite this limitation, it provides the following features (which are all also available in licensed mode):

  • Conversions from Python to q, except Python-callable objects
  • Conversions from q to Python
  • A q IPC interface

1.b Running in Licensed mode

Licensed mode is the standard way to operate PyKX, wherein it's running under a Python process with a valid q license. This modality aims to replace PyQ as the Python-first library for KX. All PyKX features are available in this mode.

The differences provided through operating with a valid kdb Insights license are:

  1. You can execute PyKX/q functionalities directly within a Python session.

  2. PyKX objects can be represented in a human readable format rather than as a memory address, namely:

    >>> kx.q('([]til 3;3?0Ng)')
pykx.Table(pykx.q('
x x1                                  
--------------------------------------
0 8c6b8b64-6815-6084-0a3e-178401251b68
1 5ae7962d-49f2-404d-5aec-f7c8abbae288
2 5a580
'))
    >>> conn.q('([]til 3;3?0Ng)')
pykx.Table._from_addr(0x7f5b72ef8860)

  3. You can analyze PyKX objects through indexing:

    >>> import pykx as kx
>>> tab = kx.q('til 10')
>>> tab[1:6]
pykx.LongVector(pykx.q('1 2 3 4 5'))
    >>> py = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> kx.toq(py)
pykx.List._from_addr(0x7fae68f00a00)
>>> kx.toq(py)[1:6]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/anaconda3/lib/python3.8/site-packages/pykx/wrappers.py", line 1166, in __getitem__
    raise LicenseException('index into K object')
pykx.exceptions.LicenseException: A valid q license must be in a known location (e.g. `$QLIC`) to index into K object.

  4. Licensed mode allows users to cast between PyKX object types. Unlicensed mode doesn't support this, showing an error as below:

    >>> import pykx as kx
>>> py = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> kx.K.cast(kx.toq(py), kx.FloatVector)
pykx.FloatVector(pykx.q('0 1 2 3 4 5 6 7 8 9f'))
    >>> import pykx as kx
>>> py = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> kx.K.cast(kx.toq(py), kx.FloatVector)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/anaconda3/lib/python3.8/site-packages/pykx/wrappers.py", line 419, in cast
    return ktype(self)
  File "/usr/local/anaconda3/lib/python3.8/site-packages/pykx/wrappers.py", line 246, in __new__
    return toq(x, ktype=None if cls is K else cls, cast=cast) # TODO: 'strict' and 'cast' flags
  File "pykx/toq.pyx", line 2543, in pykx.toq.ToqModule.__call__
  File "pykx/toq.pyx", line 470, in pykx.toq.from_pykx_k
pykx.exceptions.LicenseException: A valid q license must be in a known location (e.g. `$QLIC`) to directly convert between K types.

  5. Only licensed mode supports the classes/functionalities below. This is not an exhaustive list:

    1. kx.q.sql
    2. kx.q.read
    3. kx.q.write
    4. kx.q.schema
    5. kx.q.console
  6. Pandas API functionality for interactions with and PyKX Table objects.
  7. You can convert keyed tables to equivalent Numpy types.
  8. All types can be disambiguated, generic null can be discerned from a projection null, and similar for regular vs splayed tables.
  9. Numpy list object conversion is optimized only in licensed mode.
  10. Only licensed mode grants users access to the is_null, is_inf, has_nulls, and has_infs methods of K objects.

  11. Only licensed mode allows users to use the updated query API.

    >>> qtab.select(where=kx.Column('col2')>40)
pykx.Table(pykx.q('
col1 col2
---------
a    76
a    81
a    75
    >>> qtab.select(qtab, where='col2>40')
pykx.Table(pykx.q('
col1 col2
---------
a    76
a    81
a    75

How to choose between Licensed and Unlicensed

You can choose to initialise PyKX under one of these modes through the use of the QARGS environment variable as follows:

Mode argument Description
--unlicensed Starts PyKX in unlicensed mode. No license check is performed, and no warning is emitted at start-up if embedded q initialization fails.
--licensed Raises a PyKXException (as opposed to emitting a PyKXWarning) if embedded q initialization fails.

In addition to the PyKX specific start-up arguments, you can also use QARGS to set the standard q command-line arguments.

Alternatively, if you wish to access PyKX in unlicensed mode, you set the environment variable PYKX_UNLICENSED="true" or define this in your .pykx-config file as outlined here.

2. PyKX within q

Fully described here, the ability to use PyKX within a q session allows you to achieve the following:

  • Replace embedPy functionally with an updated, more flexible interface.
  • Use Python within a q environment without the limitations for PyKX.
  • Use Python code in conjunction with timers and subscriptions within a q/kdb+ ecosystem.

Similar to the use of PyKX in licensed mode, PyKX running under q requires a user to have access to an appropriate license containing the insights.lib.pykx and insights.lib.embedq licensing flags.