How to Work with Lists

This page introduces lists in q and shows how to create and work with them.

Lists are the core data structure in q, and they form nearly all higher-level structures. A list is an ordered collection of items, which can be either atoms or nested lists.

List types:

Simple list: Contains atoms of a uniform type (conceptually similar to a mathematical vector or a C-style array) and is stored in contiguous memory for high performance.
General lists: Can contain mixed types or nested structures. In memory, general lists are implemented as a contiguous array of pointers to their respective items.

Lists in q are dynamic; they grow by appending items to the end, functioning much like dynamically allocated arrays.

Creating a list

Simple list elements are separated by whitespace, except for symbol lists, which use the backtick prefix:

q)4 1 10            / simple list of integers (longs)
4 1 10
q)`Alice`Bob`Mike   / simple list of symbols
`Alice`Bob`Mike

General lists are defined by enclosing semicolon-separated values within parentheses. Whitespace after the separators in list definition is optional.

q

q)(42; 3.14; "string"; 1 2 3)
42
3.14
"string"
1 2 3

Python

>>>[42, 3.14, "string", [1, 2, 3]]
[42, 3.14, 'string', [1, 2, 3]]

When all items in a list are homogeneous atoms, q automatically converts the structure to a simple list.

q)(4; 1; 10)
4 1 10
q)(0b;1b;0b;1b;1b)
01011b

While type promotion provides significant storage and performance benefits, it can introduce constraints: if an outlier is deleted and the list becomes uniform, q will subsequentely prevent the insertion of non-conforming types.

q)l: (2; 3.14; 5)   / l is a general list
q)l[1]: 10
q)l                 / l was converted to a simple list
2 10 5
q)l[1]: 3.14        / strict type checking for simple lists
'type
  [0]  l[1]: 3.14
           ^

Strings are handled as lists of characters:

q)("s"; "t"; "r"; "i"; "n"; "g")
"string"

Several built-in functions facilitate list creation:

q

q)til 4         / sequence of integers [0, n)
0 1 2 3

q)4?10          / 4 random integers between 0 and 9
3 3 7 8
q)0N?til 5      / random permutation of a sequence
4 2 3 1 0

Python

>>> list(range(4))
[0, 1, 2, 3]
>>> import random
>>> [random.randint(0, 9) for _ in range(4)]
[3, 3, 7, 8]
>>> random.sample(range(5), 5)
[4, 2, 3, 1, 0]

Empty lists

An empty set of parentheses () denotes an empty general list. Empty general lists have no console display.

q)()        / empty general list
q)`int$()   / empty integer list (typed)
`int$()
q)0#0i      / same as above but using take (#) instead of cast ($)
`int$()

To visualize an empty list, use the .Q.s1 utility:

q).Q.s1 ()
"()"

Singleton lists

A singleton is a list containing exactly one item. These are created using the enlist function. Note that simply wrapping an atom in parentheses (42) does not create a list; it remains an atom.

q)enlist 42
,42
q)1#42      / using take (#) to create a singleton
,42
q)(42)
42

Projections

Omitting an element in a general list definition creates a projection, which acts as a template for future data:

q

q)f: (4; ; 10)
q)f 1
4 1 10
q)f "string"
4
"string"
10

Python

>>> f = lambda x: [4, x, 10]
>>> f(1)
[4, 1, 10]
>>> f("string")
[4, 'string', 10]

Operations

Counting and membership

The count function returns the length of a list. To check if a specific item exists within a list, use the in operator (which is vectorized).

q

q)count 4 1 10
3
q)10 in 4 1 10
1b
q)9 10 in 4 1 10    / vectorized check
01b

Python

>>> len([4, 1, 10])
3
>>> 10 in [4, 1, 10]
True
>>> [x in [4, 1, 10] for x in [9, 10]]
[False, True]

Concatenation and appending

The Join operator (,) merges two lists. You can also use the upsert keyword for more complex assignments.

q

q)4 1 10, 3 2 1
4 1 10 3 2 1
q)l: 4 1 10
q)l,: 5             / Append in-place
q)`l upsert 0 100   / Bulk append via symbol handle
`l
q)l
4 1 10 5 0 100

Python

>>> [4, 1, 10] + [3, 2, 1]
[4, 1, 10, 3, 2, 1]
>>> l = [4, 1, 10]
>>> l.append(5)
>>> l.extend([0, 100])

>>> l
[4, 1, 10, 5, 0, 100]

Lookup, indexing

Indexing in q is 0-based. Brackets are optional for simple retrieval.

q

q)l: 4 1 10
q)l[2]
10
q)l 2           / brackets not necessary for indexing
10
q)l @ 2         / using 'index at'
10
q)first l       / or l[0]
4
q)last l        / or l[-1+count l]
10
q)l 1 0         / index by list
1 4

Python

>>> l = [4, 1, 10]
>>> l[2]
10




>>> l[0]
4
>>> l[-1]
10
>>> [l[i] for i in [1, 0]]
[1, 4]

If you index out of bounds, the result is not an error. Instead you get a null value indicating "missing data".

q)l 5
0N
q)l -1
0N

For nested lists, indices can be "drilled into" using multiple brackets or a semicolon-separated list:

q)show l: (4 1;10 20)
4  1
10 20
q)l 1
10 20
q)l[1][0]
10
q)l[1;0]
10

Eliding an index in any slot is equivalent to specifying all legitimate indices for that slot:

q)l[;1]
1 20

Use index(.) to index at depth by a list

q)l . 1 0
10

Observe that the result of index retrieval has the same shape as the index. So far only atoms and lists were used for indexing, but you can go further:

q)l: 4 1 10
q)l (2 0; 1)        / index by a nested list
10 4
1
q)l ([a: 2; b:0])   / index by a dictionary
a| 10
b| 4

Indexed Assignment

You can overwrite any item by indexing:

q

q)l: 4 1 10
q)l[1]:42       / update
q)l
4 42 10
q)l[1 0]: 50
q)l
50 50 10

Python

>>> l = [4, 1, 10]
>>> l[1] = 42
>>> l
[4, 42, 10]
>>> for i in [1, 0]: l[i] = 50
>>> l
[50, 50, 10]

You can also use the general amend at. If you pass the list name as a symbol then it modifies the list in-place (otherwise returns the result):

q)l: 4 1 10
q)@[`l;1;:;42]
`l
q)l
4 42 10

Sub-lists

You can extract sub-lists using the sublist function and the take operator. The inverse operator is _:

q

q)l: 4 1 10
q)2 sublist l   / sub-list
4 1
q)-2 sublist l  / last two items
1 10
q)2 _ l         / delete first two items
,10
q)l _ 1         / delete item at index 1
4 10

Python

>>> l = [4, 1, 10]
>>> l[:2]
[4, 1]
>>> l[-2:]
[1, 10]
>>> l[2:]
[10]
>>> l[:1] + l[2:]
[4, 10]

The function take is similar to sublist but it behaves differently if the first parameter is greater than the list count:

q

q)l: 4 1 10
q)5 sublist l / capped at list length
4 1 10

q)5#l         / take 5 (wraps around if count exceeded)
4 1 10 4 1
q)5#2
2 2 2 2 2

Python

>>> l = [4, 1, 10]
>>> l[:5]
[4, 1, 10]
>>> import itertools
>>> list(itertools.islice(itertools.cycle(l), 5))
[4, 1, 10, 4, 1]
>>> 5 * [2]
[2, 2, 2, 2, 2]

Arithmetic operations

As a vector-based language, arithmetic in q is applied element-wise:

q)l: 4 1 10
q)2*l
8 2 20
q)l + 100 200 300
104 201 310
q)l = 3 1 11
010b
q)neg l
-4 -1 -10

Custom functions that use vector operations also act as vector operations and iterate over the items:

q)f: {2*x+1}
q)f l
10 4 22

There are several aggregator functions in q:

q)l: 4 1 10 2
q)sum l
17
q)avg l
4.25
q)sums l                  / running sum
4 5 15 17
q)deltas l                / deltas of adjacent items
4 -3 9 -8
q)2 msum l                / moving sum
4 5 11 12
q)0.2 0.1 0.5 0.2 wsum l  / weighted sum
6.3

Reverse lookup

You can find the first index of an item using find:

q

q)l: 4 1 10 1
q)l?1
1

Python

>>> l = [4, 1, 10, 1]
>>> l.index(1)
1

You can use the vector operation = and keyword where to find all occurrences of an item:

q

q)where l=1
1 3

Python

>>> [i for i, x in enumerate(l) if x == 1]
[1, 3]

Set operations

While q does not have a dedicated "Set" type, it provides high-performance set operations for lists:

q)l1: 1 2 3
q)l2: 2 4 6
q)l1 inter l2         / intersection
,2
q)l1 union l2         / union
1 2 3 4 6
q)l1 except l2        / difference
1 3
q)distinct 3 1 1 2 3  / unique items
3 1 2

Coalesce `^`

The coalesce operator ^ is related to , in that it employs upsert semantics to merge two lists with right prevailing over left on common keys. The difference is that null values in the right operand do not prevail over the left.

q

q)left: 30 0N 43
q)right: 10 5 0N
q)left ^ right
10 5 43

Python

>>> left  = [30, None, 43]
>>> right = [10, 5,    None]
>>> [r if r is not None else l for l, r in zip(left, right)]
[10, 5, 43]

Miscellaneous

There are several utility functions that operate on lists:

q

q)l: 4 1 10
q)reverse l         / reverse a list
10 1 4
q)asc l             / sort a list
`s#1 4 10
q)iasc l            / indices needed to sort
1 0 2

q)1 2 5 9 binr 5    / binary search
2
q)2 cut til 6       / cut into sub-lists
0 1
2 3
4 5
q)flip 2 cut til 6  / transpose
0 2 4
1 3 5
q)raze (2 1; 5 3)   / flattening
2 1 5 3

q)group `Alice`Bob`Alice`Mike`Mike
Alice| 0 2
Bob  | ,1
Mike | 3 4

q)0 1 cross 2 3     / Cartesian product
0 2
0 3
1 2
1 3

Python

>>> l = [4, 1, 10]
>>> l[::-1]
[10, 1, 4]
>>> sorted(l)
[1, 4, 10]
>>> sorted(range(len(l)), key=l.__getitem__)
[1, 0, 2]
>>> import bisect
>>> bisect.bisect_right([1, 2, 5, 9], 5) - 1
2
>>> [list(range(6))[i:i+2] for i in range(0,6,2)]
[[0, 1], [2, 3], [4, 5]]


>>> list(zip(*[list(range(6))[i:i+2] for i in range(0,6,2)]))
[(0, 2, 4), (1, 3, 5)]

>>> [x for sub in [[2,1],[5,3]] for x in sub]
[2, 1, 5, 3]

>>> from collections import defaultdict
>>> d = defaultdict(list)                       # group
>>> [d[v].append(i) for i,v in enumerate(['Alice','Bob','Alice','Mike','Mike'])]
>>> dict(d)
{'Alice': [0, 2], 'Bob': [1], 'Mike': [3, 4]}
>>> import itertools
>>> list(itertools.product([0,1],[2,3]))         # cartesian product
[(0, 2), (0, 3), (1, 2), (1, 3)]

Next Steps

For a deeper dive into lists, check out Q for Mortals §3. Lists.
Improve the performance of list operations by applying attributes.
Learn how to create dictionaries from two lists.

How to Work with Lists

Creating a list

Empty lists

Singleton lists

Projections

Operations

Counting and membership

Concatenation and appending

Lookup, indexing

Indexed Assignment

Sub-lists

Arithmetic operations

Reverse lookup

Set operations

Coalesce ^

Miscellaneous

Next Steps

Coalesce `^`