Metadata and aggregations

Properties describing a table, detection of null values and aggregations which reduce each column or row to a single value. These methods are available on pykx.Table and pykx.KeyedTable objects.

These methods follow the behavior of the pandas method of the same name, not the q function of that name. The differences from the q functions are:

Method	q function	Difference
sum, prod, min, max	sum, prd, min, max	skipna=False returns null if the column contains a null. q always skips nulls.
median	med	Nulls are skipped. q counts a null as the smallest value.
std	sdev, dev	Normalized by N - 1 by default. ddof sets the divisor to N - ddof.
count	count	Counts the values which are not null. q counts every value.
any	any	Nulls are skipped by default. q treats a null as true. A null symbol is false.
all	all	A null never makes the result false. A null symbol is false.
idxmax, idxmin	x?max x, x?min x	Returns the index label rather than the position.

sem, skew, kurt and mode have no q equivalent.

columns property

columns()

The names of the columns of the table, not including the key columns.

Examples:

>>> tab = kx.q('([] a:1 2 3; b:4 5 6)')
>>> tab.columns
pykx.SymbolVector(pykx.q('`a`b'))

dtypes property

dtypes()

The data type of each column, as a table of column names and pykx types.

Examples:

>>> tab = kx.q('([] a:1 2 3; b:4 5 6)')
>>> tab.dtypes
pykx.Table(pykx.q('
columns datatypes
---------------------
a       "kx.LongAtom"
b       "kx.LongAtom"
'))

empty property

empty()

Whether the table has no rows.

Examples:

>>> tab = kx.q('([] a:1 2 3; b:4 5 6)')
>>> tab.empty
pykx.BooleanAtom(pykx.q('0b'))
>>> kx.q('0#([] a:1 2 3)').empty
pykx.BooleanAtom(pykx.q('1b'))

ndim property

ndim()

The number of dimensions of the table, which is always 2.

Examples:

>>> tab = kx.q('([] a:1 2 3; b:4 5 6)')
>>> tab.ndim
pykx.LongAtom(pykx.q('2'))

shape property

shape()

The shape of the table as a tuple of the number of rows and the number of columns.

Examples:

>>> tab = kx.q('([] a:1 2 3; b:4 5 6)')
>>> tab.shape
(pykx.LongAtom(pykx.q('3')), pykx.LongAtom(pykx.q('2')))

size property

size()

The number of values in the table, which is rows multiplied by columns.

Examples:

>>> tab = kx.q('([] a:1 2 3; b:4 5 6)')
>>> tab.size
pykx.LongAtom(pykx.q('6'))

mean

mean(axis=0, numeric_only=False)

Returns the mean of values across the given axis.

Nulls are skipped, as they are by the q function avg.

Parameters:

Name	Type	Description	Default
axis	int	The axis to calculate the mean across. 0 is columns, 1 is rows.	0
numeric_only	bool	Include only columns or rows with numeric data.	False

Returns:

Type	Description
Dictionary	A dictionary of the mean across each row or column, keyed by row number or column name.

Examples:

>>> tab = kx.q('([] a:1 2 3; b:4 5 6)')
>>> tab.mean()
pykx.Dictionary(pykx.q('
a| 2
b| 5
'))

kurt

kurt(axis=0, numeric_only=False)

Returns the unbiased kurtosis across the given axis, using Fisher's definition.

The kurtosis of a normal distribution is 0.0. Nulls are skipped. A column with fewer than 4 non null values or with no variance gives null.

Parameters:

Name	Type	Description	Default
axis	int	The axis to calculate the kurtosis across. 0 is columns, 1 is rows.	0
numeric_only	bool	Only use columns of the table that are of a numeric data type.	False

Returns:

Type	Description
Dictionary	A dictionary of the kurtosis across each row or column, keyed by row number or column name.

Examples:

>>> tab = kx.q('([] a:1 2 4 8 16f; b:3 1 4 1 5f)')
>>> tab.kurt()
pykx.Dictionary(pykx.q('
a| 1.303763
b| -2.324219
'))

std

std(axis=0, ddof=1, numeric_only=False)

Returns the sample standard deviation across the given axis.

This is normalized by N - 1 by default, as in pandas. The q function dev is normalized by N and sdev by N - 1. Nulls are skipped and a column with N <= ddof gives null.

Parameters:

Name	Type	Description	Default
axis	int	The axis to calculate the standard deviation across. 0 is columns, 1 is rows.	0
ddof	int	Delta degrees of freedom. The divisor used is N - ddof, where N is the number of non null values.	1
numeric_only	bool	Only use columns of the table that are of a numeric data type.	False

Returns:

Type	Description
Dictionary	A dictionary of the standard deviation across each row or column, keyed by row number or column name.

Examples:

>>> tab = kx.q('([] v:1 2 3 4f)')
>>> tab.std()
pykx.Dictionary(pykx.q('
v| 1.290994
'))
>>> tab.std(ddof=0)
pykx.Dictionary(pykx.q('
v| 1.118034
'))

median

median(axis=0, numeric_only=False)

Returns the median of values across the given axis.

Nulls are skipped. The q function med counts a null as the smallest value, so the two can differ when a column contains nulls.

Parameters:

Name	Type	Description	Default
axis	int	The axis to calculate the median across. 0 is columns, 1 is rows.	0
numeric_only	bool	Include only columns or rows with numeric data.	False

Returns:

Type	Description
Dictionary	A dictionary of the median across each row or column, keyed by row number or column name.

Examples:

>>> tab = kx.q('([] a:1 0n 3f; b:4 5 6f)')
>>> tab.median()
pykx.Dictionary(pykx.q('
a| 2
b| 5
'))

skew

skew(axis=0, skipna=True, numeric_only=False)

Returns the unbiased skewness across the given axis.

A column with fewer than 3 non null values or with no variance gives null.

Parameters:

Name	Type	Description	Default
axis	int	The axis to calculate the skewness across. 0 is columns, 1 is rows.	0
skipna	bool	Ignore any null values along the axis.	True
numeric_only	bool	Only use columns of the table that are of a numeric data type.	False

Returns:

Type	Description
Dictionary	A dictionary where the key is the column name or row number and the value is the result of skew on that column or row.

Examples:

>>> tab = kx.q('([] a:1 2 4 8 16f; b:3 1 4 1 5f)')
>>> tab.skew()
pykx.Dictionary(pykx.q('
a| 1.325315
b| 0.05240784
'))

mode

mode(axis=0, numeric_only=False, dropna=True)

Returns the mode of values across the given axis.

Parameters:

Name	Type	Description	Default
axis	int	The axis to calculate the mode across. 0 is columns, 1 is rows.	0
numeric_only	bool	Include only columns or rows with numeric data.	False
dropna	bool	Remove null values from the data before calculating the mode.	True

Returns:

Type	Description
Table	A table of the modes of each row or column. Columns with fewer modes are padded with nulls.

Examples:

>>> tab = kx.q('([] a:1 1 2; b:3 4 4)')
>>> tab.mode()
pykx.Table(pykx.q('
a b
---
1 4
'))

sem

sem(axis=0, ddof=1, numeric_only=False)

Returns the unbiased standard error of the mean across the given axis.

This is normalized by N - 1 by default. Nulls are skipped.

Parameters:

Name	Type	Description	Default
axis	int	The axis to calculate the standard error across. 0 is columns, 1 is rows.	0
ddof	int	Delta degrees of freedom. The divisor used is N - ddof, where N is the number of non null values.	1
numeric_only	bool	Only use columns of the table that are of a numeric data type.	False

Returns:

Type	Description
Dictionary	A dictionary of the standard error of the mean across each row or column, keyed by row number or column name.

Examples:

>>> tab = kx.q('([] a:1 2 4 8 16f; b:3 1 4 1 5f)')
>>> tab.sem()
pykx.Dictionary(pykx.q('
a| 2.727636
b| 0.8
'))

abs

abs(numeric_only=False)

Take the absolute value of each element of the table.

Parameters:

Name	Type	Description	Default
numeric_only	bool	Only use columns whose data type has an absolute value. Otherwise any other column raises an error.	False

Returns:

Type	Description
Table or KeyedTable	A table or keyed table of the same shape with only positive values.

Examples:

>>> tab = kx.q('([] a:-1 2 -3)')
>>> tab.abs()
pykx.Table(pykx.q('
a
-
1
2
3
'))

all

all(axis=0, bool_only=False, skipna=True)

Returns whether all values across the given axis are truthy.

This follows the behavior of the pandas method of the same name, not the q function all. Zero is false and any other number is true. Nulls are skipped, so they never make all false. A null symbol is an empty string, which is false.

Parameters:

Name	Type	Description	Default
axis	int	The axis to calculate all across. 0 is columns, 1 is rows.	0
bool_only	bool	Only use columns of the table that are boolean types.	False
skipna	bool	Ignore any null values along the axis. With skipna=False a null counts as true.	True

Returns:

Type	Description
Dictionary	A dictionary where the key is the column name or row number and the value is the result of all on that column or row.

Examples:

>>> tab = kx.q('([] a:1 0n 3f; b:0 1 0; c:`x``z; d:101b)')
>>> tab.all()
pykx.Dictionary(pykx.q('
a| 1
b| 0
c| 0
d| 0
'))

any

any(axis=0, bool_only=False, skipna=True)

Returns whether any value across the given axis is truthy.

This follows the behavior of the pandas method of the same name, not the q function any. Zero is false and any other number is true. A null is skipped by default, whereas q treats a null as nonzero. A null symbol is an empty string, which is false.

Parameters:

Name	Type	Description	Default
axis	int	The axis to calculate any across. 0 is columns, 1 is rows.	0
bool_only	bool	Only use columns of the table that are boolean types.	False
skipna	bool	Ignore any null values along the axis. With skipna=False a null counts as true.	True

Returns:

Type	Description
Dictionary	A dictionary where the key is the column name or row number and the value is the result of any on that column or row.

Examples:

>>> tab = kx.q('([] a:0 0n 0f; b:0 1 0)')
>>> tab.any()
pykx.Dictionary(pykx.q('
a| 0
b| 1
'))
>>> tab.any(skipna=False)
pykx.Dictionary(pykx.q('
a| 1
b| 1
'))

max

max(axis=0, skipna=True, numeric_only=False)

Returns the maximum value across the given axis.

This follows the behavior of the pandas method of the same name, not the q function max. The q function always skips nulls.

Parameters:

Name	Type	Description	Default
axis	int	The axis to calculate the maximum across. 0 is columns, 1 is rows.	0
skipna	bool	Ignore any null values along the axis. With skipna=False a column containing a null returns a null.	True
numeric_only	bool	Only use columns of the table that are of a numeric data type.	False

Returns:

Type	Description
Dictionary	A dictionary where the key is the column name or row number and the value is the result of max on that column or row.

Examples:

>>> tab = kx.q('([] a:1 0n 3f; b:4 5 6f)')
>>> tab.max()
pykx.Dictionary(pykx.q('
a| 3
b| 6
'))
>>> tab.max(skipna=False)
pykx.Dictionary(pykx.q('
a|
b| 6
'))

min

min(axis=0, skipna=True, numeric_only=False)

Returns the minimum value across the given axis.

This follows the behavior of the pandas method of the same name, not the q function min. The q function always skips nulls.

Parameters:

Name	Type	Description	Default
axis	int	The axis to calculate the minimum across. 0 is columns, 1 is rows.	0
skipna	bool	Ignore any null values along the axis. With skipna=False a column containing a null returns a null.	True
numeric_only	bool	Only use columns of the table that are of a numeric data type.	False

Returns:

Type	Description
Dictionary	A dictionary where the key is the column name or row number and the value is the result of min on that column or row.

Examples:

>>> tab = kx.q('([] a:1 0n 3f; b:4 5 6f)')
>>> tab.min()
pykx.Dictionary(pykx.q('
a| 1
b| 4
'))
>>> tab.min(skipna=False)
pykx.Dictionary(pykx.q('
a|
b| 4
'))

idxmax

idxmax(axis=0, skipna=True, numeric_only=False)

Returns the index of the first occurrence of the maximum over the given axis.

The result is an index label, as in pandas, and not a position. For a table with no key it is the row number and for a keyed table it is the value of the key column. Under groupby the labels are those of the original table. A column of only nulls gives null.

Parameters:

Name	Type	Description	Default
axis	int	The axis to calculate idxmax across. 0 is columns, 1 is rows.	0
skipna	bool	Ignore any null values along the axis.	True
numeric_only	bool	Only use columns of the table that are of a numeric data type.	False

Returns:

Type	Description
Dictionary	A dictionary where the key is the column name or row number and the value is the result of idxmax on that column or row.

Examples:

>>> tab = kx.q('1!([] id:10 20 30; v:5 9 2)')
>>> tab.idxmax()
pykx.Dictionary(pykx.q('
v| 20
'))

idxmin

idxmin(axis=0, skipna=True, numeric_only=False)

Returns the index of the first occurrence of the minimum over the given axis.

The result is an index label, as in pandas, and not a position. For a table with no key it is the row number and for a keyed table it is the value of the key column. Under groupby the labels are those of the original table. A column of only nulls gives null.

Parameters:

Name	Type	Description	Default
axis	int	The axis to calculate idxmin across. 0 is columns, 1 is rows.	0
skipna	bool	Ignore any null values along the axis.	True
numeric_only	bool	Only use columns of the table that are of a numeric data type.	False

Returns:

Type	Description
Dictionary	A dictionary where the key is the column name or row number and the value is the result of idxmin on that column or row.

Examples:

>>> tab = kx.q('1!([] id:10 20 30; v:5 9 2)')
>>> tab.idxmin()
pykx.Dictionary(pykx.q('
v| 30
'))

prod

prod(axis=0, skipna=True, numeric_only=False, min_count=0)

Returns the product of all values across the given axis.

This follows the behavior of the pandas method of the same name, not the q function. Nulls are skipped unless skipna=False.

Parameters:

Name	Type	Description	Default
axis	int	The axis to calculate the product across. 0 is columns, 1 is rows.	0
skipna	bool	Ignore any null values along the axis. With skipna=False a column containing a null returns a null.	True
numeric_only	bool	Only use columns of the table that are of a numeric data type.	False
min_count	int	If not 0, a null is returned when there are fewer than min_count non null values.	0

Returns:

Type	Description
Dictionary	A dictionary where the key is the column name or row number and the value is the result of prod on that column or row.

Examples:

>>> tab = kx.q('([] a:1 0n 3f; b:4 5 6f)')
>>> tab.prod()
pykx.Dictionary(pykx.q('
a| 3
b| 120
'))
>>> tab.prod(skipna=False)
pykx.Dictionary(pykx.q('
a|
b| 120
'))

sum

sum(axis=0, skipna=True, numeric_only=False, min_count=0)

Returns the sum of all values across the given axis.

This follows the behavior of the pandas method of the same name, not the q function sum. The q function always skips nulls, so use skipna=False or the query API if a null should be kept.

Parameters:

Name	Type	Description	Default
axis	int	The axis to calculate the sum across. 0 is columns, 1 is rows.	0
skipna	bool	Ignore any null values along the axis. With skipna=False a column containing a null returns a null.	True
numeric_only	bool	Only use columns of the table that are of a numeric data type.	False
min_count	int	If not 0, a null is returned when there are fewer than min_count non null values.	0

Returns:

Type	Description
Dictionary	A dictionary where the key is the column name or row number and the value is the result of sum on that column or row.

Examples:

>>> tab = kx.q('([] a:1 0n 3f)')
>>> tab.sum()
pykx.Dictionary(pykx.q('
a| 4
'))
>>> tab.sum(skipna=False)
pykx.Dictionary(pykx.q('
a|
'))

agg

agg(func, axis=0, *args, **kwargs)

Aggregate the table using one or more operations over an axis.

A dictionary for func is not supported on a groupby and agg is not supported on a KeyedTable.

Parameters:

Name	Type	Description	Default
func	callable, str, list or dict	A function, the name of a function, a list of either, or a dictionary of column names to functions or function names.	required
axis	int	Only 0, which applies the operation to each column, is supported.	0
*args	Any	Positional arguments passed to func.	()
**kwargs	Any	Keyword arguments passed to func.	{}

Returns:

Type	Description
Dictionary, Table or KeyedTable	The result of each operation as a dictionary or table.

Examples:

>>> tab = kx.q('([] a:1 2 3; b:4 5 6)')
>>> tab.agg('sum')
pykx.Dictionary(pykx.q('
a| 6
b| 15
'))
>>> tab.agg(['min', 'max'])
pykx.KeyedTable(pykx.q('
function| a b
--------| ---
min     | 1 4
max     | 3 6
'))

count

count(axis=0, numeric_only=False)

Returns the count of non null values across the given axis.

This counts the values which are not null. The q function count counts every value, including nulls.

Parameters:

Name	Type	Description	Default
axis	int	The axis to count elements across. 0 is columns, 1 is rows.	0
numeric_only	bool	Only use columns of the table that are of a numeric data type.	False

Returns:

Type	Description
Dictionary	A dictionary where the key is the column name or row number and the value is the result of count on that column or row.

Examples:

>>> tab = kx.q('([] a:1 0n 3f; b:4 5 6f)')
>>> tab.count()
pykx.Dictionary(pykx.q('
a| 2
b| 3
'))

isna

isna()

Detect null values in the table.

Returns:

Type	Description
Table	A table of the same shape where 1b marks a null value and 0b a value which is not.

Examples:

>>> tab = kx.q('([] a:1 0n 3f; b:4 5 6f)')
>>> tab.isna()
pykx.Table(pykx.q('
a b
---
0 0
1 0
0 0
'))

fills

fills()

Fill each null value with the last value before it which is not null.

Returns:

Type	Description
Table	A table with the nulls filled forward.

Examples:

>>> tab = kx.q('([] a:1 0n 0n 4f)')
>>> tab.fills()
pykx.Table(pykx.q('
a
-
1
1
1
4
'))

ungroup

ungroup()

Expand a table with list valued columns into one row per list item.

Returns:

Type	Description
Table	The ungrouped table, as in the q function ungroup.

Examples:

>>> tab = kx.q('([] k:`a`b; v:(1 2;3 4 5))')
>>> tab.ungroup()
pykx.Table(pykx.q('
k v
---
a 1
a 2
b 3
b 4
b 5
'))

isnull

isnull()

Detect null values in the table. This is an alias of isna.

Returns:

Type	Description
Table	A table of the same shape where 1b marks a null value and 0b a value which is not.

Examples:

>>> tab = kx.q('([] a:1 0n 3f; b:4 5 6f)')
>>> tab.isnull()
pykx.Table(pykx.q('
a b
---
0 0
1 0
0 0
'))

notna

notna()

Detect values in the table which are not null.

Returns:

Type	Description
Table	A table of the same shape where 1b marks a value which is not null and 0b a null.

Examples:

>>> tab = kx.q('([] a:1 0n 3f; b:4 5 6f)')
>>> tab.notna()
pykx.Table(pykx.q('
a b
---
1 1
0 1
1 1
'))

notnull

notnull()

Detect values in the table which are not null. This is an alias of notna.

Returns:

Type	Description
Table	A table of the same shape where 1b marks a value which is not null and 0b a null.

Examples:

>>> tab = kx.q('([] a:1 0n 3f; b:4 5 6f)')
>>> tab.notnull()
pykx.Table(pykx.q('
a b
---
1 1
0 1
1 1
'))