Skip to content

Changelog

PyKX 1.0.2

Deprecations & Removals

  • PyKX has been removed from PyPI. It is now only available from the KX Nexus server. The installation instructions have been updated to reflect this.

PyKX 1.0.1

Deprecations & Removals

  • The sync parameter for pykx.QConnection and pykx.QConnection.__call__ has been renamed to the less confusing name wait. The sync parameter remains, but its usage will result in a DeprecationWarning being emitted. The sync parameter will be removed in a future version.

Features & Fixes

  • Updated to stable classifier (Development Status :: 5 - Production/Stable) in project metadata. Despite this update being done in version 1.0.1, version 1.0.0 is still the first stable release of PyKX.
  • PyKX now provides source distributions (sdist). It can be downloaded from PyPI using pip download --no-binary=:all: --no-deps pykx. As noted in the installation docs, installations built from the source will only receive support on a best-effort basis.
  • Fixed Pandas NaT conversion to q types. Now pykx.toq(pandas.NaT, ktype=ktype) produces a null temporal atom for any given ktype (e.g. pykx.TimeAtom).
  • Added a doc page for limitations of embedded q.
  • Added a test to ensure large vectors are correctly handled (5 GiB).
  • Always use synchronous queries internally, i.e. fix QConnection(sync=False).
  • Disabled the context interface over IPC. This is a temporary measure that will be reversed once q function objects are updated to run in the environment they were defined in by default.
  • Reduced the time it takes to import PyKX. There are plans to reduce it further, as import pykx remains fairly slow.
  • Updated to KXI Core 2.1 & rename qce -> kxic.
  • Misc test updates.
  • Misc doc updates.

PyKX 1.0.0

Migration Notes

To switch from Pykdb to PyKX, you will need to update the name of the dependency from pykdb to pykx in your pyproject.toml/requirements.txt/setup.cfg/etc. When Pykdb was renamed to PyKX, its version number was reset. The first public release of PyKX has the version number 1.0.0, and will employ semantic versioning.

Pay close attention to the renames listed below, as well as the removals. Many things have been moved to the top-level, or otherwise reorganized. A common idiom with Pykdb was the following:

from pykdb import q, k

It is recommended that the following be used instead:

import pykx as kx

This way the many attributes at the top-level can be easily accessed without any loss of context, for example:

kx.q # Can be called to execute q code
kx.K # Base type for objects in q; can be used to convert a Python object into a q type
kx.SymbolAtom # Type for symbol atoms; can be used to convert a `str` or `bytes` into a symbol atom
kx.QContext # Represents a q context via the PyKX context interface
kx.QConnection # Can be called to connect to a q process via q IPC
kx.PyKXException # Base exception type for exceptions specific to PyKX and q
kx.QError # Exception type for errors that occur in q
kx.LicenseException # Exception type raised when features that require a license are used without
kx.QHOME # Path from which to load q files, set by $QHOME environment variable
kx.QARGS # List of arguments provided to the embedded q instance at startup, set by $QARGS environment variable
# etc.

You can no longer rely on the context being reset to the global context after each call into embedded q, however IPC calls are unaffected.

Renames

  • Pykdb has been renamed to PyKX. Pykdb -> PyKX; PYKDB -> PYKX; pykdb -> pykx.
  • The adapt module has been renamed to toq, and it can be called directly. Instead of pykdb.adapt.adapt(x) one should write pykx.toq(x).
  • The k module has been renamed to wrappers. All wrapper classes can be accessed from the top-level, i.e. pykx.K, pykx.SymbolAtom, etc.
  • The "module interface" (pykdb.module_interface) has been renamed to the "context interface" (pykx.ctx). All pykx.Q instances (i.e. pykx.q and all pykx.QConnection instances) have a ctx attribute, which is the global QContext for that pykx.Q instance. Usually, one need not directly access the global context. Instead, one can access its subcontexts directly e.g. q.dbmaint instead of q.ctx.dbmaint.
  • KdbError (and its subclasses) have been renamed to QError
  • pykdb.ctx.KdbContext has been renamed to pykx.ctx.QContext, and is available from the top-level, i.e. pykx.QContext.
  • The Connection class in the IPC module has been renamed to QConnection, and is now available at the top-level, i.e. pykx.QConnection.
  • The q type wrapper DynamicLoad has been renamed to Foreign (pykdb.k.DynamicLoad -> pykx.Foreign).

Deprecations & Removals

  • The pykdb.q.ipc attribute has been removed. The IPC module can be accessed directly instead at pykx.ipc, but generally one will only need to access the QConnection class, which can be accessed at the top-level: pykx.QConnection.
  • The pykdb.q.K attribute has been removed. Instead, K types can be used as constructors for that type by leveraging the toq module. For example, instead of pykdb.q.K(x) one should write pykx.K(x). Instead of pykx.q.K(x, k_type=pykx.k.SymbolAtom) one should write pykx.SymbolAtom(x) or pykx.toq(x, ktype=pykx.SymbolAtom).
  • Most KdbError/QError subclasses have been removed, as identifying them is error prone, and we are unable to provide helpful error messages for most of them.
  • The pykx.kdb singleton class has been removed.

Dependencies

  • More Numpy, Pandas, and PyArrow versions are supported. Current pandas~=1.0, numpy~=1.20,<1.22, and pyarrow>=3.0.0 are supported. PyArrow remains an optional dependency.
  • A dependency on find-libpython~=0.2 was added. This is only used when running PyKX under a q process (see details in the section below about new alpha features).
  • A dependency on the system library libcurl was added for Linux. This dependency will be made optional in a future release.

Features & Fixes

  • The pykx.Q class has been added as the base class for pykx.EmbeddedQ (the class for pykx.q) and pykx.QConnection.
  • The pykx.EmbeddedQ process now perisists its context between calls.
  • The console now works over IPC.
  • The query module now works over IPC. Because K objects hold no reference to the q instance that created them (be it local or over IPC), K tables no longer have select/exec/update/delete methods with themselves projected in as the first argument. That is to say, instead of writing t.select(...), write q.qsql.select(t, ...), where q is either pykx.q or an instance of pykx.QConnection, and t was obtained from q.
  • The context interface now works over IPC.
  • Nulls and infinities are now handled as nulls and infinities, rather than as their underlying values. pykx.Atom.is_null, pykx.Atom.is_inf, pykx.Collection.has_nulls, and pykx.Collection.has_infs have been added. Numpy, Pandas, and PyArrow handles integral nulls with masked arrays, and they handle temporal nulls with NaT. NaN continues to be used for real/float nulls. The general Python representation (from .py()) uses K objects for nulls and infinities.
  • Calling bool on pykx.K objects now either raises a TypeError, or return the unambiguously correct result. For ambiguous cases such as pykx.Collection instances, use .any(), .all(), or a length check instead.
  • Assignment to q reserved words or the q context now raises a pykx.PyKXException.
  • pykx.toq.from_list (previously pykdb.adapt.adapt_list) now works in unlicensed mode.
  • q.query and q.sql are now placeholders (set to None). The query interface can be accessed from q.qsql.
  • Ternary pow now raises TypeError for RealNumericVector and RealNumericAtom.
  • QContext objects are now context handlers, e.g. with pykx.q.dbmaint: # operate in .dbmaint within this block. This context handler supports arbitrary nesting.
  • __getitem__ now raises a pykx.LicenseException when used in unlicensed mode. Previously it worked for a few select types only. If running in unlicensed mode, one should perform all q indexing in the connected q process, and all Python indexing after converting the K object to a Python/Numpy/Pandas/PyArrow object.
  • pykx.QConnection (previously pykdb.ipc.Connection) objects now have an informative/idiomatic repr.
  • Calls to pykx.q now support up to 8 arguments beyond the required query at position 0, similar to calling pykx.QConnection instances. These arguments are applied to the result of the query.
  • Embedded q is now used to count the number of rows a table has.
  • All dynamic linking to libq and libe has been replaced by dynamic loading. As a result, the modules previously known as adapt and adapt_unlicensed have been unified under pykx.toq.
  • PyKX now attempts to initialize embedded q when pykx is imported, rather than when pykx.q is first accessed. As a result, the error-prone practice of supplying the pykx.kdb singleton class with arguments for embedded q is now impossible.
  • Arguments for embedded q can now be supplied via the environment variable $QARGS in the form of command-line arguments. For example, QARGS='--unlicensed' causes PyKX to enter unlicensed mode when it is started, and QARGS='-o 8' causes embedded q to use an offset from UTC of 8 hours. These could be combined as QARGS='--unlicensed -o 8'.
  • Added the --licensed startup flag (to be provided via the $QARGS environment variable), which can be used to raise a pykx.PyKXException (rather than emitting a warning) if PyKX fails to start in licensed mode (likely because of a missing/invalid q license).
  • PyKX Linux wheels are now PEP 600 compliant, built to the manylinux_2_17 standard.
  • Misc other bug fixes.
  • Misc doc improvements.

Performance Improvements

  • Converting nested lists from q to Python is much faster.
  • Internally, PyKX now calls q functions with arguments directly instead of creating a pykx.Function instance then calling it. This results in modest performance benefits in some cases.
  • The context interface no longer loads every element of a context when the context is first accessed, thereby removing the computation spike, which could be particularly intense for large q contexts.

New Alpha Features

Alpha features are subject to change

Alpha features are not stable will be subject to changes without notice. Use at your own risk.

  • q can now load PyKX by loading the q file pykx.q. pykx.q can be copied into $QHOME by running pykx.install_into_QHOME(). When loaded into q, it will define the .pykx namespace, which notably has .pykx.exec and .pykx.pyeval. This allows for Python code to be run within q libraries and applications without some of the limitations of embedded q such as the lack of the q main loop, or the lack of timers. When q loads pykx.q, it attempts to source the currently active Python environment by running python, then fetching the environment details from it.