# How to work with lists

_This page introduces lists in q and shows how to create and work with them._

Lists are the core data structure in q, and they form nearly all higher-level structures. A list is an ordered collection of _items_, which can be either [atoms](atoms.md) or nested lists.

List types:

- **Simple list**: Contains atoms of a uniform type (conceptually similar to a mathematical vector or a C-style array) and is stored in contiguous memory.
- **General lists**: Can contain mixed types or nested structures. In memory, general lists are implemented as a contiguous array of pointers to their respective items.

Lists in q are dynamic; they grow by appending items to the end, functioning much like dynamically allocated arrays.

## List creation

Simple list elements are separated by whitespace, except for symbol lists, which use the backtick prefix:
```q
q)4 1 10            / simple list of integers (longs)
4 1 10
q)`Alice`Bob`Mike   / simple list of symbols
`Alice`Bob`Mike
```

Define a general list by enclosing semicolon-separated values in parentheses. Whitespace after the separators in a list definition is optional.


**q**
```q
q)(42; 3.14; "string"; 1 2 3)
42
3.14
"string"
1 2 3
```


**Python**
```python
>>> [42, 3.14, "string", [1, 2, 3]]
[42, 3.14, 'string', [1, 2, 3]]
```


When all items in a list are homogeneous atoms, q automatically converts the structure to a _simple list_.
```q
q)(4; 1; 10)
4 1 10
q)(0b;1b;0b;1b;1b)
01011b
```

Type promotion saves storage, because a simple list holds values rather than pointers, but it introduces a constraint: if you delete an outlier and the list becomes uniform, q rejects non-conforming types.
```q
q)l: (2; 3.14; 5)   / l is a general list
q)l[1]: 10
q)l                 / l was converted to a simple list
2 10 5
q)l[1]: 3.14        / strict type checking for simple lists
'type
  [0]  l[1]: 3.14
           ^
```

q handles strings as lists of characters:
```q
q)("s"; "t"; "r"; "i"; "n"; "g")
"string"
```

Several built-in functions help you create lists:


**q**
```q
q)til 4         / sequence of integers [0, n)
0 1 2 3

q)4?10          / 4 random integers between 0 and 9
3 3 7 8
q)0N?til 5      / random permutation of a sequence
4 2 3 1 0
```


**Python**
```python
>>> list(range(4))
[0, 1, 2, 3]
>>> import random
>>> [random.randint(0, 9) for _ in range(4)]
[3, 3, 7, 8]
>>> random.sample(range(5), 5)
[4, 2, 3, 1, 0]
```


### Empty lists

An empty set of parentheses `()` denotes an empty general list. Empty general lists have no console display.
```q
q)()        / empty general list
q)`int$()   / empty integer list (typed)
`int$()
q)0#0i      / same as above but using take (#) instead of cast ($)
`int$()
```

To visualize an empty list, use the [`.Q.s1`](../../../ref/dotq.md#s1-string-representation) utility:
```q
q).Q.s1 ()
"()"
```

### Singleton lists

A _singleton_ is a list containing exactly one item. Use [`enlist`](../../../ref/enlist.md) to create one. Wrapping an atom in parentheses `(42)` does not create a list; it remains an atom.
```q
q)enlist 42
,42
q)1#42      / using take (#) to create a singleton
,42
q)(42)
42
```

### Projections

Omitting an element in a general list definition creates a projection, which acts as a template for future data:


**q**
```q
q)f: (4; ; 10)
q)f 1
4 1 10
q)f "string"
4
"string"
10
```


**Python**
```python
>>> f = lambda x: [4, x, 10]
>>> f(1)
[4, 1, 10]
>>> f("string")
[4, 'string', 10]
```


## Operations

### Counting and membership

The [count](../../../ref/count.md) function returns the length of a list. To check whether a specific item exists within a list, use the [in](../../../ref/in.md) operator (which is vectorized).


**q**
```q
q)count 4 1 10
3
q)10 in 4 1 10
1b
q)9 10 in 4 1 10    / vectorized check
01b
```


**Python**
```python
>>> len([4, 1, 10])
3
>>> 10 in [4, 1, 10]
True
>>> [x in [4, 1, 10] for x in [9, 10]]
[False, True]
```


### Concatenation and appending

The Join operator ([`,`](../../../ref/join.md)) merges two lists. You can also use the [upsert](../../../ref/upsert.md) keyword for more complex assignments.


**q**
```q
q)4 1 10, 3 2 1
4 1 10 3 2 1
q)l: 4 1 10
q)l,: 5             / Append in place
q)`l upsert 0 100   / Bulk append via symbol handle
`l
q)l
4 1 10 5 0 100
```


**Python**
```python
>>> [4, 1, 10] + [3, 2, 1]
[4, 1, 10, 3, 2, 1]
>>> l = [4, 1, 10]
>>> l.append(5)
>>> l.extend([0, 100])

>>> l
[4, 1, 10, 5, 0, 100]
```


### Lookup and indexing

Indexing in q is 0-based. Brackets are optional when you index with a single value.


**q**
```q
q)l: 4 1 10
q)l[2]
10
q)l 2           / brackets not necessary for indexing
10
q)l @ 2         / using 'index at'
10
q)first l       / or l[0]
4
q)last l        / or l[-1+count l]
10
q)l 1 0         / index by list
1 4
```


**Python**
```python
>>> l = [4, 1, 10]
>>> l[2]
10


>>> l[0]
4
>>> l[-1]
10
>>> [l[i] for i in [1, 0]]
[1, 4]
```


If you index out of bounds, the result is not an error. Instead you get a null value, which indicates missing data.
```q
q)l 5
0N
q)l -1
0N
```

For nested lists, drill into indices with multiple brackets or a semicolon-separated list:
```q
q)show l: (4 1;10 20)
4  1
10 20
q)l 1
10 20
q)l[1][0]
10
q)l[1;0]
10
```

Eliding an index in any slot is equivalent to specifying all legitimate indices for that slot:
```q
q)l[;1]
1 20
```

Use [index](../../../ref/apply.md#index) (`.`) to index at depth by a list:
```q
q)l . 1 0
10
```

Observe that the result of index retrieval has the same shape as the index. So far you have indexed with atoms and lists only, but you can go further:
```q
q)l: 4 1 10
q)l (2 0; 1)        / index by a nested list
10 4
1
q)l ([a: 2; b:0])   / index by a dictionary
a| 10
b| 4
```

### Indexed assignment

You can overwrite any item by indexing:


**q**
```q
q)l: 4 1 10
q)l[1]:42       / update
q)l
4 42 10
q)l[1 0]: 50
q)l
50 50 10
```


**Python**
```python
>>> l = [4, 1, 10]
>>> l[1] = 42
>>> l
[4, 42, 10]
>>> for i in [1, 0]: l[i] = 50
>>> l
[50, 50, 10]
```


You can also use the [general amend at](../../../ref/amend.md). If you pass the list name as a symbol, it modifies the list in place; otherwise it returns the result:
```q
q)l: 4 1 10
q)@[`l;1;:;42]
`l
q)l
4 42 10
```

### Sublists

You can extract sublists using the [sublist](../../../ref/sublist.md) function and the [take](../../../ref/take.md) operator. The inverse operator is [`_` Drop](../../../ref/drop.md):


**q**
```q
q)l: 4 1 10
q)2 sublist l   / sublist
4 1
q)-2 sublist l  / last two items
1 10
q)2 _ l         / delete first two items
,10
q)l _ 1         / delete item at index 1
4 10
```


**Python**
```python
>>> l = [4, 1, 10]
>>> l[:2]
[4, 1]
>>> l[-2:]
[1, 10]
>>> l[2:]
[10]
>>> l[:1] + l[2:]
[4, 10]
```


The function [take](../../../ref/take.md) is similar to `sublist` but it behaves differently if the first parameter is greater than the list count:


**q**
```q
q)l: 4 1 10
q)5 sublist l / capped at list length
4 1 10

q)5#l         / take 5 (wraps around if count exceeded)
4 1 10 4 1
q)5#2
2 2 2 2 2
```


**Python**
```python
>>> l = [4, 1, 10]
>>> l[:5]
[4, 1, 10]
>>> import itertools
>>> list(itertools.islice(itertools.cycle(l), 5))
[4, 1, 10, 4, 1]
>>> 5 * [2]
[2, 2, 2, 2, 2]
```


### Arithmetic operations

q is a vector language, so it applies arithmetic element-wise:
```q
q)l: 4 1 10
q)2*l
8 2 20
q)l + 100 200 300
104 201 310
q)l = 3 1 11
010b
q)neg l
-4 -1 -10
```

Custom functions that use vector operations also act as vector operations and iterate over the items:
```q
q)f: {2*x+1}
q)f l
10 4 22
```

q provides several aggregators:
```q
q)l: 4 1 10 2
q)sum l
17
q)avg l
4.25
q)sums l                  / running sum
4 5 15 17
q)deltas l                / deltas of adjacent items
4 -3 9 -8
q)2 msum l                / moving sum
4 5 11 12
q)0.2 0.1 0.5 0.2 wsum l  / weighted sum
6.3
```

### Iteration

As the previous section shows, a function built from [atomic](../../../ref/atomic.md) operations already iterates over a list on its own. Use explicit iteration when the function is not atomic — one with a side effect, or one that needs to see a whole item at a time. [`each`](../../../ref/each.md) applies such a function item by item:


**q**
```q
q)l: 4 1 10
q){show "the item is ", string x} each l;
"the item is 4"
"the item is 1"
"the item is 10"
```


**Python**
```python
>>> l = [4, 1, 10]
>>> for item in l:
...     print("the item is", item)
the item is 4
the item is 1
the item is 10
```


> **Note — Do not reach for `each` when the function already iterates**
>
>
> Applying an atomic function directly is both shorter and faster. Write `f l`, not `` f each l ``.
>
To see each item together with its index, apply a binary function with the [each-both](../../../ref/maps.md#each) iterator (`'`):


**q**
```q
q){string[x], ": ", string y}'[til count l; l] / or (til count l) {...}' l
"0: 4"
"1: 1"
"2: 10"
```


**Python**
```python
>>> [f"{i}: {v}" for i, v in enumerate(l)]
['0: 4', '1: 1', '2: 10']
```


To carry a running result along the list, use the [accumulators](../../../ref/accumulators.md) [over](../../../ref/over.md) (`/`) for the final value and [scan](../../../ref/over.md) (`\`) for every intermediate one. Supply a starting value on the left where you need one:


**q**
```q
q)(+/) l          / same as sum l
15

q)(+\) l          / same as sums l
4 5 15
q){x+2*y}/[0; l]  / fold with a starting value
30
```


**Python**
```python
>>> import functools, itertools
>>> functools.reduce(lambda a, b: a+b, l)
15
>>> list(itertools.accumulate(l))
[4, 5, 15]
>>> functools.reduce(lambda a, b: a+2*b, l, 0)
30
```


### Reverse lookup

You can find the first index of an item using [`find`](../../../ref/find.md):


**q**
```q
q)l: 4 1 10 1
q)l?1
1
```


**Python**
```python
>>> l = [4, 1, 10, 1]
>>> l.index(1)
1
```


You can use the vector operation `=` and keyword [`where`](../../../ref/where.md) to find all occurrences of an item:


**q**
```q
q)where l=1
1 3
```


**Python**
```python
>>> [i for i, x in enumerate(l) if x == 1]
[1, 3]
```


If the list is sorted, you can use binary search and functions [bin and binr](../../../ref/bin.md).
```q
q)1 2 5 9 bin 5   / binary search
2
```

Find (`?`), `bin`, and `binr` implicitly iterate over the right parameter if a list is provided.

### Set operations

q has no dedicated set type, but it provides set operations for lists:
```q
q)l1: 1 2 3
q)l2: 2 4 6
q)l1 inter l2         / intersection
,2
q)l1 union l2         / union
1 2 3 4 6
q)l1 except l2        / difference
1 3
q)distinct 3 1 1 2 3  / unique items
3 1 2
```

### Null replacement with Coalesce (`^`)

The [coalesce](../../../ref/coalesce.md) operator `^` uses upsert semantics to merge two lists, with the right prevailing over the left on common keys. The difference from `,` is that null values in the right operand do not prevail over the left.


**q**
```q
q)left: 30 0N 43
q)right: 10 5 0N
q)left ^ right
10 5 43
```


**Python**
```python
>>> left  = [30, None, 43]
>>> right = [10, 5,    None]
>>> [r if r is not None else l for l, r in zip(left, right)]
[10, 5, 43]
```


### Reshaping, sorting, and grouping

Several utility functions operate on lists:


**q**
```q
q)l: 4 1 10
q)reverse l         / reverse a list
10 1 4
q)asc l             / sort a list
`s#1 4 10
q)iasc l            / indices needed to sort
1 0 2

q)2 cut til 6       / cut into sublists
0 1
2 3
4 5
q)flip 2 cut til 6  / transpose
0 2 4
1 3 5
q)raze (2 1; 5 3)   / flattening
2 1 5 3

q)group `Alice`Bob`Alice`Mike`Mike
Alice| 0 2
Bob  | ,1
Mike | 3 4

q)0 1 cross 2 3     / Cartesian product
0 2
0 3
1 2
1 3
```


**Python**
```python
>>> l = [4, 1, 10]
>>> l[::-1]
[10, 1, 4]
>>> sorted(l)
[1, 4, 10]
>>> sorted(range(len(l)), key=l.__getitem__)
[1, 0, 2]
>>> import bisect
>>> bisect.bisect_right([1, 2, 5, 9], 5) - 1
2
>>> [list(range(6))[i:i+2] for i in range(0,6,2)]
[[0, 1], [2, 3], [4, 5]]


>>> list(zip(*[list(range(6))[i:i+2] for i in range(0,6,2)]))
[(0, 2, 4), (1, 3, 5)]

>>> [x for sub in [[2,1],[5,3]] for x in sub]
[2, 1, 5, 3]

>>> from collections import defaultdict
>>> d = defaultdict(list)                       # group
>>> [d[v].append(i) for i,v in enumerate(['Alice','Bob','Alice','Mike','Mike'])]
>>> dict(d)
{'Alice': [0, 2], 'Bob': [1], 'Mike': [3, 4]}
>>> import itertools
>>> list(itertools.product([0,1],[2,3]))         # cartesian product
[(0, 2), (0, 3), (1, 2), (1, 3)]
```


## Next steps

- Improve the performance of list operations by applying [attributes](../../../ref/attr.md).
- Learn how to create [dictionaries](dictionaries.md) from two lists.
