Jupyter q Magic Command Notebook¶

The purpose of this notebook is to exemplify how to use the q Magic command in a Jupyter notebook.

The Jupyter q magic command in PyKX allows you to execute q code within a Jupyter notebook. It provides seamless integration with the q programming language.

This example Notebook has the following sections:

1. Import PyKX
2. Create the external q process
3. Execute against Embedded q
4. SQL interface
5. q namespaces
6. (Advanced) q over IPC

1. Import PyKX¶

To run this example, first import the PyKX library:

In [2]:

import pykx as kx

import pykx as kx

2. Create the external q process¶

You can run an external q process by using the following Python code:

In [3]:

import subprocess
import time

try:
    with kx.PyKXReimport():
        proc = subprocess.Popen(
            ('q', '-p', '5000')
        )
    time.sleep(2)
except:
    raise kx.QError('Unable to create q process on port 5000')

import subprocess import time try: with kx.PyKXReimport(): proc = subprocess.Popen( ('q', '-p', '5000') ) time.sleep(2) except: raise kx.QError('Unable to create q process on port 5000')

Or execute this command in a terminal:

$ q -p 5000

3. Execute against Embedded q¶

To execute q code within PyKX's EmbeddedQ module, type %%q at the beginning of the cell:

In [4]:

%%q
til 10

%%q til 10

0 1 2 3 4 5 6 7 8 9

After %%q you can further add two execution options:

Execution option	Description
--debug	Prints the q backtrace before raising a QError if the cell gives an error.
--display	Calls display rather than the default print on returned objects.

In [5]:

%%q
([] a: 1 2 3)

%%q ([] a: 1 2 3)

a
-
1
2
3

In [6]:

%%q --display
([] a: 1 2 3)

%%q --display ([] a: 1 2 3)

	a
0	1
1	2
2	3

4. SQL interface¶

The s) syntax runs SQL queries against local tables within the q process.

Note: To use the SQL interface, first you need to load the s.k_ library.

In [7]:

%%q
\l s.k_
tab:([]a:1000?1000; b:1000?500.0; c:1000?`AAPL`MSFT`GOOG);
s) select * from tab where a>500 and b<250.0 limit 5

%%q \l s.k_ tab:([]a:1000?1000; b:1000?500.0; c:1000?`AAPL`MSFT`GOOG); s) select * from tab where a>500 and b<250.0 limit 5

a   b        c   
-----------------
916 149.4238 AAPL
898 97.74763 MSFT
900 45.35602 MSFT
677 150.6862 MSFT
741 151.1293 MSFT

5. q namespaces¶

You can use q namespaces, and switch between them with \d.

Note: The namespace is reset back to the base namespace . between cells.

In [8]:

%%q
\d .example
f: {[x] til x};

%%q \d .example f: {[x] til x};

In [9]:

%%q
\d
.example.f[10]

%%q \d .example.f[10]

.example
0 1 2 3 4 5 6 7 8 9

6. (Advanced) q over IPC¶

After %%q you can include connection information, if you wish to connect to a remote q process over IPC.

The list of supported connection parameters is below. The rule is:

• If they have a type, it must be followed by a second value/parameter.
• If there's no type after them, you can use them as a standalone flag.

Parameter	Object type and description
--host	(string) The host to connect to.
--port	(integer) The port to connect over.
--user	(string) The username to use when connecting.
--password	(string) The password to use when connecting.
--timeout	(float) The time in seconds before the query times out. Defaults to no timeout.
--nolarge	Disable messages over 2GB being sent / received.
--tls	Use a tls connection.
--unix	Use a unix connection.
--reconnection_attempts	(integer) How many reconnection attempts to make.
--noctx	Disable the context interface.

Connect to a q server running on localhost at port 5000 as user using password password and disable the context interface.

In [10]:

%%q --host localhost --port 5000 --user user --pass password --noctx
til 10

%%q --host localhost --port 5000 --user user --pass password --noctx til 10

0 1 2 3 4 5 6 7 8 9

All connection arguments are optional, except the --port argument. If --host is not provided localhost is the default host.

In [11]:

%%q --port 5000
tab:([]a:1000?1000; b:1000?500.0; c:1000?`AAPL`MSFT`GOOG);

%%q --port 5000 tab:([]a:1000?1000; b:1000?500.0; c:1000?`AAPL`MSFT`GOOG);

Note that it's possible to execute q code spanning multiple lines:

In [12]:

%%q --port 5000
afunc: {[x; y]
  x + y  
  };
afunc[0; 1]
afunc[2; 3]

%%q --port 5000 afunc: {[x; y] x + y }; afunc[0; 1] afunc[2; 3]

1
5

In [13]:

# Shutdown the q process we were connected to for the IPC demo
proc.kill()

# Shutdown the q process we were connected to for the IPC demo proc.kill()