From Kx Wiki
Jump to: navigation, search

301 Permanent move

This article is now at

Please update your bookmark. The old wiki will remain here for a while. If you prefer it to the new format, please tell the Librarian why.


PyQ: Python Integration



PyQ provides seamless integration of Python and Q code. It brings Python and Q interpretors in the same process and allows code written in either of the languages to operate on the same data. In PyQ, Python and Q objects live in the same memory space and share the same data.

The following annotated session demonstrates some of the features:

$ pyq
>>> from pyq import q
>>> = q('([]date:();sym:();qty:())')
>>> q.insert('trade', (date(2006,10,6), 'IBM', 200))
>>> q.insert('trade', (date(2006,10,6), 'MSFT', 100))
date       sym  qty
2006.10.06 IBM  200
2006.10.06 MSFT 100
>>> query = q('{[s;d]select from trade where sym=s,date=d}')
>>> query('IBM', date(2006,10,6))
date       sym  qty
2006.10.06 IBM 200


           pip install -i --no-binary pyq pyq

Use and redistribution are subject to a license.

On-Line Documentation

Documentation is available from the python prompt

>>> help('pyq')

or at the PyQ homepage.

Performance Features

Q is much faster than Python. The following table shows times of adding 10,000 integers in four diffrent ways: (1) using a Q function on Q data, (2) using a Q function on Python data, (3) using a Python function on Python data, and (for completeness) (4) using a Python function on Q data.

Setup Call Time
x = q.til(10000) q.sum(x) 31.5 usec
x = range(10000) q.sum(x) 187 usec
x = range(10000) sum(x) 648 usec
x = q.til(10000) sum(x) 849 usec

One-way conversion

The best performance is achieved when Q functions are used on Q data. PyQ is designed to take advantage of this fact. Python data is automatically converted to Q, when mixed with Q data and stays in Q form unless converted back explicitly:

>>> q('1 2 3') + [3,2,1]
k('4 4 4')
>>> list(_)
[4, 4, 4]

Compiled Queries

Parsing large queries can be slow. PyQ makes it easy to pre-compile Q queries and reuse them as if they were Python functions.

>>> query = q('{[s;d]select from trade where sym=s,date=d}')

PyQ queries combine the features of Python and Q functions. Like Python functions, they can be called with named arguments,

>>> query(d=date(2006,10,6),s='IBM')

and, like Q functions, they can be partially applied:

>>> q1 = query(d=date(2006,10,6))

resulting in a query with fewer parameters,

>>> q1('MSFT')

Writing a Kdb+ Client

PyQ is designed primarily for embeded in-memory databases. For maximum performance one should avoid sending data between processes or over network. Nevertheless, client programming is very simple with PyQ.

>>> from qc import *
>>> c = Client(port=1234)
>>> c.insert('t', (1,'x'))
>>> c.insert('t', (2,'y'))
>>> c.insert('t', (3,'z'))
a b
1 x
2 y
3 z
>>> c('{select from t where a>x}'), 1)
k('+`a`b!(2 3;`y`z)')
Personal tools