Modes of operation

This page explains how to operate KDB-X Python in Python and q, with or without a KDB-X license.

KDB-X Python can operate in different modes. Each mode has its limitations and requirements, so we're breaking them down into the following:

KDB-X Python within Python
- 1.a Unlicensed
- 1.b Licensed
KDB-X Python within q with a KDB-X license

1. KDB-X Python within Python

The purpose of operating KDB-X Python within a Python session is to replace qPython and PyQ. Within Python, KDB-X Python has two modes of operation:

Licensed (this means you have a KDB-X license with KDB-X Python enabled)
Unlicensed (this means you don't have a KDB-X license or a license in which KDB-X Python is not enabled)

The main difference between the two is that the Unlicensed mode is for IPC-based communication. This mean that it allows to create IPC connections and convert data from Pythonic representations to pykx objects.

The following table outlines more key differences:

Feature	Licensed	Unlicensed
Convert objects from q to Pythonic types and vice-versa
Query synchronously and asynchronously a q server via IPC
Query synchronously and asynchronously a q server with TLS enabled
Interact with `pykx` tables via a Pandas like API
Run arbitrary q code within a Python session
Display `pykx`/q objects within a Python session
Load KDB-X Databases within a Python session
Read/write JSON, CSV and q formats to/from disk
Access to Python classes for SQL, schema creation, custom data conversion
Run Python within a q session using KDB-X Python under q
Full support for nulls, infinities, data slicing and casting
Production support

1.a Running in Unlicensed mode

Unlicensed mode is a feature-limited mode of operation for KDB-X Python. Its aim is to replace qPython, which has the benefit of not requiring a valid q license (except for the q license required to run the remote q process that KDB-X Python connects to in this mode).

This mode cannot run q embedded within it. Also, it lacks the ability to run q code within the local Python process or any functionality that depends on running q code. Despite this limitation, it provides the following features (which are all also available in licensed mode):

Conversions from Python to q, except Python-callable objects
Conversions from q to Python
A q IPC interface

1.b Running in Licensed mode

Licensed mode is the standard way to operate KDB-X Python, wherein it's running under a Python process with a valid q license. This modality aims to replace PyQ as the Python-first library for KX. All KDB-X Python features are available in this mode.

The differences provided through operating with a valid KDB-X license are:

You can execute KDB-X Python/q functionalities directly within a Python session.

pykx objects can be represented in a human readable format rather than as a memory address, namely:

Licensed modeUnlicensed mode

>>> kx.q('([]til 3;3?0Ng)')
pykx.Table(pykx.q('
x x1                                  
--------------------------------------
0 8c6b8b64-6815-6084-0a3e-178401251b68
1 5ae7962d-49f2-404d-5aec-f7c8abbae288
2 5a580
'))

>>> conn.q('([]til 3;3?0Ng)')
pykx.Table._from_addr(0x7f5b72ef8860)

You can analyze pykx objects through indexing:

Licensed modeUnlicensed mode

>>> import pykx as kx
>>> tab = kx.q('til 10')
>>> tab[1:6]
pykx.LongVector(pykx.q('1 2 3 4 5'))

>>> py = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> kx.toq(py)
pykx.List._from_addr(0x7fae68f00a00)
>>> kx.toq(py)[1:6]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/anaconda3/lib/python3.8/site-packages/pykx/wrappers.py", line 1166, in __getitem__
    raise LicenseException('index into K object')
pykx.exceptions.LicenseException: A valid q license must be in a known location (e.g. `$QLIC`) to index into K object.

Licensed mode allows users to cast between pykx object types. Unlicensed mode doesn't support this, showing an error as below:

Licensed modeUnlicensed mode

>>> import pykx as kx
>>> py = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> kx.K.cast(kx.toq(py), kx.FloatVector)
pykx.FloatVector(pykx.q('0 1 2 3 4 5 6 7 8 9f'))

>>> import pykx as kx
>>> py = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> kx.K.cast(kx.toq(py), kx.FloatVector)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/anaconda3/lib/python3.8/site-packages/pykx/wrappers.py", line 419, in cast
    return ktype(self)
  File "/usr/local/anaconda3/lib/python3.8/site-packages/pykx/wrappers.py", line 246, in __new__
    return toq(x, ktype=None if cls is K else cls, cast=cast) # TODO: 'strict' and 'cast' flags
  File "pykx/toq.pyx", line 2543, in pykx.toq.ToqModule.__call__
  File "pykx/toq.pyx", line 470, in pykx.toq.from_pykx_k
pykx.exceptions.LicenseException: A valid q license must be in a known location (e.g. `$QLIC`) to directly convert between K types.

Only licensed mode supports the classes/functionalities below. This is not an exhaustive list:
1. kx.q.sql
2. kx.q.read
3. kx.q.write
4. kx.q.schema
5. kx.q.console
Pandas API functionality for interactions with and pykx Table objects.
You can convert keyed tables to equivalent Numpy types.
All types can be disambiguated, generic null can be discerned from a projection null, and similar for regular vs splayed tables.
Numpy list object conversion is optimized only in licensed mode.
Only licensed mode grants users access to the is_null, is_inf, has_nulls, and has_infs methods of K objects.

Only licensed mode allows users to use the updated query API.

Licensed ModeUnlicensed Mode

>>> qtab.select(where=kx.Column('col2')>40)
pykx.Table(pykx.q('
col1 col2
---------
a    76
a    81
a    75

>>> qtab.select(qtab, where='col2>40')
pykx.Table(pykx.q('
col1 col2
---------
a    76
a    81
a    75

How to choose between Licensed and Unlicensed

You can choose to initialise KDB-X Python under one of these modes through the use of the QARGS environment variable as follows:

Mode argument	Description
`--unlicensed`	Starts KDB-X Python in unlicensed mode. No license check is performed, and no warning is emitted at start-up if embedded q initialization fails.
`--licensed`	Raises a `PyKXException` (as opposed to emitting a `PyKXWarning`) if embedded q initialization fails.

In addition to the KDB-X Python specific start-up arguments, you can also use QARGS to set the standard q command-line arguments.

Alternatively, if you wish to access KDB-X Python in unlicensed mode, you set the environment variable PYKX_UNLICENSED="true" or define this in your config-pykx file as outlined here.

2. KDB-X Python within q

Fully described here, the ability to use KDB-X Python within a q session allows you to achieve the following:

Replace embedPy functionally with an updated, more flexible interface.
Use Python within a q environment without the limitations for KDB-X Python.
Use Python code in conjunction with timers and subscriptions within a q/KDB-X ecosystem.

Similar to the use of KDB-X Python in licensed mode, KDB-X Python running under q requires a user to have access to an appropriate license containing the insights.lib.pykx and insights.lib.embedq (or py and eq) licensing flags.