Compression and encryption API

This page documents utilities for managing configuration of compression and encryption settings for on-disk data.

CompressionAlgorithm

Bases: Enum

The compression algorithm used when compressing a DB partition/column.

Supported algorithms are qipc, gzip, snappy, lz4hc and zstd. These algorithms support different compression levels denoting the agressivness of compression in each case.

algorithm	levels
none	0
q IPC	0
gzip	0-9
snappy	0
lz4hc	0-16
zstd	-7-22

Encrypt

Encrypt(path=None, password=None)

A class for controlling the use of encryption with PyKX.

Parameters:

Name	Type	Description	Default
path	str	Location of a user's encryption key file	None
password	str	Password for encryption file	None

Returns:

Type	Description
None	A None object on successful invocation

>>> import pykx as kx
>>> encrypt = kx.Encrypt('/path/to/mykey.key', 'mySuperSecretPassword')

load_key

load_key()

Load the encyption key from the file given during class initialization. This overwrites the master key in the embedded q process. See here for details.

Returns:

Type	Description
None	A None object on successful invocation

>>> import pykx as kx
>>> encrypt = kx.Encrypt('/path/to/mykey.key', 'mySuperSecretPassword')
>>> encrypt.load_key()

Compress

Compress(algo=CompressionAlgorithm.none, block_size=2**17, level=None)

A class object for controlling q compression with PyKX.

Parameters:

Name	Type	Description	Default
algo	CompressionAlgorithm	Compression algorithm to use. This must be one of: kx.CompressionAlgorithm.none kx.CompressionAlgorithm.ipc kx.CompressionAlgorithm.gzip kx.CompressionAlgorithm.snappy kx.CompressionAlgorithm.lz4hc	CompressionAlgorithm.none
block_size	int	Must be a power of 2 between 12 and 20 denoting the pageSize or allocation granularity to 1MB. Read compression parameters for more information.	2**17
level	int	Compression level for the algo parameter. Algorithms that support non-zero values have higher compression ratios as the provided level increases.	None

Returns:

Type	Description
None	A None object on successful invocation

>>> import pykx as kx
>>> comp = kx.Compress(kx.CompressionAlgorithm.gzip, level=5)

global_init

global_init(encrypt=False)

Globally initialise compression settings. Once run, using kx.q.set to persist data to disk compresses the data based on specified compression settings. Refer to compression by default for more details.

Parameters:

Name	Type	Description	Default
encrypt	bool	A kx.Encrypt object denoting if and using what credentials encryption is to be applied.	False

Returns:

Type	Description
None	A None object on successful invocation

>>> import pykx as kx
>>> comp = kx.Compress(kx.CompressionAlgorithm.gzip, level=2)
>>> kx.q.z.zd
pykx.Identity(pykx.q('::'))
>>> comp.global_init()
>>> kx.q.z.zd
pykx.LongVector(pykx.q('17 2 2'))