Skip to content

Remote Function Execution

This page explains how to execute Python functions on q servers in PyKX.

Remote Functions let you define Python functions within your Python environment which can interact with kdb+ data on a q process. Once defined, these functions are registered to a remote session object along with any Python dependencies which need to be imported. The remote session object establishes and manages the remote connection to the kdb+/q server.

To execute kdb+/q functions using PyKX, go to PyKX under q

Requirements and limitations

Before you start:

  • Make sure all necessary Python requirements are installed on the client server. For this functionality you need dill>=0.2.
  • Confirm that the kdb+/q server you connect to can load PyKX under q.
  • Ensure that you have the correct versions of Python library dependencies in your kdb+/q environment at runtime.
  • Run the following command:
pip install pykx[remote]

Functional walkthrough

This walkthrough demonstrates the following steps:

  1. Initialize a q/kdb+ server loading PyKX under q on a specified port.
  2. Import PyKX and generate a remote session object which denotes the process against which the Python functions will be executed.
  3. Define a number of Python functions which will be executed on the remote q/kdb+ server.

Initializea q/kdb+ server with PyKX under q

This step ensures you have a q process running with PyKX under q, as well as having a kdb+ table available to query. If you have this already, proceed to the next step.

Ensure that you have q installed. If you do not have this installed please follow the guide provided here, retrieving your license following the instructions provided here.

Install PyKX under q using the following command.

python -c "import pykx;pykx.install_into_QHOME()"

Start the q process to which you will execute your functions.

q pykx.q -p 5050

Create a table which you will use within your Python analytics defined below.

q)N:1000
q)tab:([]sym:N?`AAPL`MSFT`GOOG`FDP;price:100+N?100f;size:10+N?100)

Set a requirement for users to provide a username/password if you wish to add security to your q process.

.z.pw:{[u;p]$[(u~`user)&p~`password;1b;0b]}

Import PyKX and create a session

Create a session object from a Python environment of your choice, which establishes and manages the remote connection to the kdb+/q server.

>>> import pykx as kx
>>> session = kx.remote.session(host='localhost', port=5050, username='user', password='password')

Define and execute Python functions using a session

Tag the Python functions you want to run on the remote server using the kx.remote.function decorator. This registers the functions on the session object you have just created. 

>>> @kx.remote.function(session)
... def zero_arg_function():
...     return 10
>>> zero_arg_function()
pykx.LongAtom(pykx.q('10'))
>>> @kx.remote.function(session)
... def single_arg_function(x):
...     return x+10
>>> single_arg_function(10)
pykx.LongAtom(pykx.q('20'))
>>> @kx.remote.function(session)
... def multi_arg_function(x, y):
...     return x+y
>>> multi_arg_function(10, 20)
pykx.LongAtom(pykx.q('30'))

Add any Python libraries which need to be available when executing the function(s) you have just defined. You can achieve this in three ways:

  1. Adding the libraries keyword when generating your session object
  2. Using session.libraries on an existing session to import required libraries before defining your function
  3. Importing libraries within the body of the function being executed

Examples of each of these methods can be seen below:

>>> import pykx as kx
>>> session = kx.remote.session(port=5050, libraries={'kx': pykx})
>>> session.libraries({'np': 'numpy', 'kx': 'pykx'})
>>> @function(session)
... def dependent_function(x, y, z):
...     return kx.q.mavg(4, np.linspace(x, y, z))
>>> dependent_function(0, 10, 10)
pykx.FloatVector(pykx.q('0 0.5555556 1.111111 2.222222 3...'))
>>> @function(remote_session)
... def dependent_function(x, y, z):
...     import pykx as kx
...     import numpy as np
...     return kx.q.mavg(4, np.linspace(x, y, z))
>>> dependent_function(0, 10, 10)
    pykx.FloatVector(pykx.q('0 0.5555556 1.111111 2.222222 3...'))

While both are valid, we suggest using libraries as a method or keyword as it allows for pre-checking of the libraries prior to definition of the function and will be expanded over time to include additional validation.