PyKX installation guide

This page explains how to install PyKX on your machine.

Pre-requisites

Before you start, make sure you have:

Python (versions 3.8-3.12)
pip

Recommended: a virtual environment with packages such as venv from the standard library.

Supported environments

KX only supports versions of PyKX built by KX (installed from wheel files) for:

Linux (manylinux_2_17_x86_64, linux-arm64) with CPython 3.8-3.12
macOS (macosx_10_10_x86_64, macosx_10_10_arm) with CPython 3.8-3.12
Windows (win_amd64) with CPython 3.8-3.12

We provide assistance to user-built installations of PyKX only on a best-effort basis.

1. Install PyKX

You can install PyKX from three sources:

Install PyKX from PyPIInstall PyKX from AnacondaInstall PyKX from GitHub

Ensure you have a recent version of pip:

pip install --upgrade pip

Then install the latest version of PyKX with the following command:

pip install pykx

For Linux x86 and arm-based architectures, you can install PyKX from the kx channel on Anaconda as follows:

conda install -c kx pykx

Type y when prompted to accept the installation.

Clone the PyKX repository:

git clone https://github.com/kxsystems/pykx

Enter the cloned repository and install PyKX using pip:

cd pykx
pip install .

At this point you have partial access to PyKX. To gain access to all PyKX features, follow the steps in the next section, otherwise go straight to 3. Verify PyKX Installation.

2. Install a kdb Insights license

To use all PyKX functionalities, you need to download and install a kdb Insights license.

Legacy kdb+/q licenses do not support all PyKX features.

There are two types of kdb Insights licenses for PyKX: personal and commercial. For either of them, you have two installation options:

a) from Python
b) using environment variables

2.a Install license in Python

Follow the steps below to install a kdb Insights license for PyKX from Python:

Start your Python session:
```
$ python
```

Import the PyKX library. When prompted to accept the installation, type Y or press Enter:

>>> import pykx as kx

Thank you for installing PyKX!

We have been unable to locate your license for PyKX. Running PyKX in unlicensed mode has reduced functionality.
Would you like to install a license? [Y/n]:

Indicate whether you have access to an existing PyKX enabled license or not, type N or press Enter to continue with accessing a new license:
```
Do you have access to an existing license for PyKX that you would like to use? [N/y]:
```

Choose whether you wish to install a personal or commercial license, type Y or press Enter to choose a personal license

Is the intended use of this software for:
    [1] Personal use (Default)
    [2] Commercial use
Enter your choice here [1/2]:

When asked if you would like to apply for a license, type Y or press Enter:

Personal licenseCommercial license

To apply for a PyKX license, navigate to https://kx.com/kdb-insights-personal-edition-license-download
Shortly after you submit your license application, you will receive a welcome email containing your license information.
Would you like to open this page? [Y/n]:

To apply for your PyKX license, contact your KX sales representative or sales@kx.com.
Alternately apply through https://kx.com/book-demo.
Would you like to open this page? [Y/n]:

For personal use, complete the form to receive your welcome email. For commercial use, the license will be provided over email after the commercial evaluation process has been followed with the support of your sales representative.

Choose the desired method to activate your license by typing 1, 2, or 3 as appropriate:

Select the method you wish to use to activate your license:
    [1] Download the license file provided in your welcome email and input the file path (Default)
    [2] Input the activation key (base64 encoded string) provided in your welcome email
    [3] Proceed with unlicensed mode
Enter your choice here [1/2/3]:

Depending on your choice (1, 2, or 3), complete the installation by following the final step as below:

123

Personal licenseCommercial license

Provide the download location of your license (for example, ~/path/to/kc.lic):

Provide the download location of your license (for example, ~/path/to/k4.lic):

Provide your activation key (base64 encoded string) provided with your welcome email:

No further actions needed.

Validate the correct installation of your license:

>>> kx.q.til(10)
pykx.LongVector(pykx.q('0 1 2 3 4 5 6 7 8 9'))

2.b Install license with environment variables

For environment-specific flexibility, there are two ways to install your license: by using a file or by copying text. Both are sourced in your welcome email. Click on the tabs below, read the instructions, and choose the method you wish to follow:

Using a fileUsing text

For personal usage, navigate to the personal license and complete the form. For commercial usage, contact your KX sales representative or sales@kx.com or apply through https://kx.com/book-demo.
On receipt of an email from KX, download and save the license file to a secure location on your computer.
Set an environment variable pointing to the folder with the license file. (Learn how to set environment variables from here).
- Variable Name: QLIC
- Variable Value: /user/path/to/folder

For personal usage, navigate to the personal license and complete the form. For commercial usage, contact your KX sales representative or sales@kx.com or apply through https://kx.com/book-demo.
On receipt of an email from KX, copy the base64 encoded contents of your license provided in plain-text within the email.
On your computer, set an environment variable KDB_LICENSE_B64 when using a personal license or KDB_K4LICENSE_B64 for a commercial license, pointing with the value copied in step 2. (Learn how to set environment variables from here).
- Variable Name: KDB_LICENSE_B64 / KDB_K4LICENSE_B64
- Variable Value: <copied contents from email>

To validate if you successfully installed your license with environment variables, start Python and import PyKX as follows:

$ python
>>> import pykx as kx
>>> kx.q.til(5)
pykx.LongVector(pykx.q('0 1 2 3 4'))

Tip: automatic license renewal setup

When your license nears its expiry date, you can set PyKX to automatically renew it. To do this, modify the environment variable KDB_LICENSE_B64 or KDB_K4LICENSE_B64 with your new license information. When PyKX initializes with the expired license, it will attempt to overwrite it with the new value:

$python
>>> import pykx as kx
Initialisation failed with error: exp
Your license has been updated using the following information:
Environment variable: 'KDB_K4LICENSE_B64'
License write location: /user/path/to/license/k4.lic

3. Verify PyKX installation

To verify if you successfully installed PyKX on your system, run:

python -c"import pykx;print(pykx.__version__)"

This command should display the installed version of PyKX.

Dependencies

Expand for Required and Optional PyKX dependencies

Required

PyKX depends on the following third-party Python packages:

pandas>=1.2, <2.0; python_version=='3.8'
pandas>=1.2, <=2.2.3; python_version>'3.8'
numpy~=1.22, <2.0; python_version<'3.11'
numpy~=1.23, <2.0; python_version=='3.11'
numpy~=1.26, <2.0; python_version=='3.12'
pytz>=2022.1
toml~=0.10.2
dill>=0.2.0
requests>=2.25.0

Note: All are installed automatically by pip when you install PyKX.

Here's a breakdown of how PyKX uses these libraries:
- NumPy: converts data from PyKX objects to NumPy equivalent Array/Recarray style objects; direct calls to NumPy functions such as numpy.max with PyKX objects relies on the NumPy Python API.
- Pandas: converts PyKX data to Pandas Series/DataFrame equivalent objects or to PyArrow data formats. Pandas is used as an intermediary data format.
- pytz: converts data with timezone information to PyKX objects to ensure that the offsets are accurately applied.
- toml: for configuration parsing and management, with .pykx-config as outlined here.
- dill: use in the serialization and deserialization of Python objects when interfacing between kdb+ and Python processes using remote functions or real-time capture functionality.

Optional

Optional Python dependencies:

pyarrow >=3.0.0: install pyarrow extra, for example pip install pykx[pyarrow].
find-libpython ~=0.2: install debug extra, for example pip install pykx[debug].
ast2json ~=0.3: install with dashboards extra, for example pip install pykx[dashboards]
dill >=0.2: install via pip, with remote extra, for example pip install pykx[remote]
beautifulsoup4 >=4.10.0: install with help extra, for example pip install pykx[help]
markdown2 >=2.5.0: install with help extra, for example pip install pykx[help]
psutil >=5.0.0: install via pip, with streaming extra, for example pip install pykx[streaming]

Here's a breakdown of how PyKX uses these libraries:

PyArrow: converts PyKX objects to and from their PyArrow equivalent table/array objects.
find-libpython: provides the libpython.{so|dll|dylib} file required by PyKX under q.
ast2json: required for KX Dashboards Direct integration.
psutil: facilitates the stopping and killing of a q process on a specified port allowing for orphaned q processes to be stopped, functionality defined here.

Optional non-Python dependencies:

libssl for TLS on IPC connections.
libpthread on Linux/MacOS when using the PYKX_THREADING environment variable.

Troubleshooting and Support

If you encounter any issues during the installation process, refer to the following sources for assistance:

Visit our troubleshooting guide.
Ask a question on the KX community at learninghub.kx.com.
Use Stack Overflow and tag pykx or kdb depending on the subject.
Go to support.

Optional: Installing a q executable

The following section is optional and primarily required if you are looking to make use of the Real-Time Capture functionality provided by PyKX.

Do I need a q executable?

For the majority of functionality provided by PyKX you do not explicitly need access to a q executable. Users within a Python process who do not have a q executable will be able to complete tasks such as the following:

Convert data to/from Python types
Run analytics on in-memory and on-disk databases
Create databases
Query remote q/kdb+ processes via IPC
Execute numpy functions with PyKX data

If however you need to make use of the Real-Time Capture functionality you will need access to a q executable. Fundamentally the capture and persistence of real-time data and the application of analytics on this streaming data is supported via deployment of code on q processes.

Configuring PyKX to use an existing executable

By default when attempting to start a q process for use within the Real-Time Capture workflows PyKX will attempt to call q directly, this method however is not fully reliable when using the Python subprocess module. As such the following setup can be completed to point more explicitly at your executable.

If you already have a q executable PyKX can use this when initializing the Real-Time Capture APIs through the setting of the following in you configuration file or as environment variables

Variable	Explanation
`PYKX_Q_EXECUTABLE`	Specifies the location of the q executable which should be called. Typically this will be `QHOME/[lmw]64/q[.exe]`
`QHOME`	The directory to which q was installed

Installing an executable

Installing using PyKX

For users who do not have access to a q executable, PyKX provides a utility function kx.util.install_q to allow users access q locally.

The following default information is used when installing the q executable:

Parameter	Default	Explanation
location	`'~/q'` or `'C:\q'`	The location to which q will be installed if not otherwise specified.
date	`'2024.07.08'`	The dated version of kdb+ 4.0 which is to be installed.

The following provide a number of examples of using the installation functionality under various conditions.

Installing to default location
```
>>> kx.util.install_q()
```
Installing to a specified location
```
>>> kx.util.install_q('~/custom')
```

Installation of q via this method will update the configuration file .pykx-config at either ~/q or C:\q to include the location of QHOME and PYKX_Q_EXECUTABLE to be used.

Installing without PyKX

The installed q executable is not required to be installed via PyKX. If you wish to install q following the traditional approach you can follow the install instructions outlined here or through signing up for a free-trial here.

Verify PyKX can use the executable

Verifying that PyKX has access to the executable can be done through execution of the function kx.util.start_q_subprocess and requires either your configuration file or environment variables to include PYKX_Q_EXECUTABLE. This is outlined here.

>>> import pykx as kx
>>> server = kx.util.start_q_subprocess(5052)
>>> conn = kx.SyncQConnection(port=5052)     # Connect to subprocess
>>> conn('1+1')
pykx.LongAtom(pykx.q('2'))
>>> server.kill()

Next steps

That's it! You can now start using PyKX in your Python projects: