EmbedPy provides an approach for using Python from q, but it does not provide a way to interface with q from Python. The EmbedPy interface was designed specifically for q developers who wish to leverage functionality in Python which is not immediately/easily available to q developers. This includes but is not limited to Machine Learning functionality, statistical methods, and plotting.
PyQ brings the Python and q interpreters into the same process so that code written in either of the languages operates on the same data. Unfortunately to use PyQ one must execute the PyQ binary, or start PyQ from q. This makes PyQ unsuitable for most Python use-cases which require the use of a Python binary. It is not possible to start a Python process, and then import PyQ.
Because of this, it is impossible to develop Python software that depends on PyQ, unless you are willing to run it in a different process. This barrier reasonably makes Python developers hesitant to use PyQ, as it locks them into using the PyQ binary to execute their program.
Like PyKX, qPython takes a Python-first approach, but unlike PyKX it works entirely over IPC. Python objects being sent to q and q objects being returned are serialized, sent over a socket, and then deserialized. While this is a common use case for many users, it is a very expensive process both in terms of processing time and memory usage. For many users wishing to use q data within Python for analysis this overhead can be limiting.
PyKX supports two principal use cases:
- It allows users to store, query, manipulate, and use q objects within a Python process.
- It also allows users to query external q processes via an IPC interface.
PyKX takes a much more Pythonic approach to interfacing between Python and q than is offered by PyQ. An example would be that PyKX imposes a class-based hierarchical type system onto the q type system, unlike PyQ which exposes all q objects as
K objects with no subclasses.
Furthermore, PyKX provides a context interface which can be used to load q scripts, and interact with q contexts and contexts in a Pythonic manner.
The querying functionality provided with PyKX when compared to PyQ is similar in so far as they provide keyword arguments for the by/where clauses but in PyKX this is wrapped around the q functional select/update/delete/exec functions rather than generating the qSQL query. This has benefits when dealing with the update functions in particular in so far as python data can be used for the updates within the function calls directly.
Fundamentally the IPC interfacing that is provided by this interface when compared to qPython is very different. qPython reads and converts data directly from the socket, whereas PyKX leverages the q memory space embedded within Python to store the data for later conversion within a referenced location in that memory space.
This has a major benefits with regard to the flexibility of the solution and supported conversions. All conversions that are supported by the interface (
.pa()) are supported by the IPC interface in the presence or absence of a license. Similarly all these types can be passed as arguments to the variadic synchronous and asynchronous functions which expands the usability of the interface to a broader user base.
Similarly to the qPython interface PyKX will allow users to make use of the IPC functionality in the absence of a license. As such users who do not need the full power of the interface or who wish to limit the number of licenses being used can run the interface to retrieve data.