How to work with dictionaries¶

This page explains how to create and work with dictionaries in q.

A dictionary is an association between a list of keys and a list of values. You can also think of it as key-value pairs, but q always stores it as a pair of lists.

Dictionary creation¶

A dictionary maps a list of keys to a corresponding list of values. The two lists must have the same count, and the key list must be a unique collection. k!v returns a dictionary, where k is the key and v is the value. Use key and value to decompose a dictionary into its key and value lists.

Example:

q

q)show d1:`Alice`Bob`Mike!30 25 43
Alice| 30
Bob  | 25
Mike | 43
q)key d1
`Alice`Bob`Mike
q)value d1
30 25 43
q)type d1
99h

Python

>>> d1 = dict(zip(['Alice', 'Bob', 'Mike'], [30, 25, 43]))
>>> d1
{'Alice': 30, 'Bob': 25, 'Mike': 43}

>>> d1.keys()
dict_keys(['Alice', 'Bob', 'Mike'])
>>> d1.values()
dict_values([30, 25, 43])
>>> type(d1)
<class 'dict'>

Note

Dictionary keys should be unique (no duplicates), but q raises no error if duplicates are present. However, operations on dictionaries with duplicate keys are undefined.

Both the keys and the values can be nested:

q)(`a`b; 1 2 3; "Alice")!3 cut til 8
`a`b   | 0 1 2
1 2 3  | 3 4 5
"Alice"| 6 7

If the keys are known to be unique, you can apply the u attribute to them, as below. The dictionary then functions as a hash table, and indexing is faster. The hash table uses storage and adds some overhead to the creation.

q)show d1:(`u#`Alice`Bob`Mike)!30 25 43
Alice| 30
Bob  | 25
Mike | 43
q)key d1
`u#`Alice`Bob`Mike

Dictionary literal syntax¶

kdb+ 4.1 introduced dictionary literal syntax. You can concisely define dictionaries if the keys are symbols. It presents the mapping in a more human-readable way:

q

q)([Alice: 30; Bob: 25; Mike: 43])
Alice| 30
Bob  | 25
Mike | 43

Python

>>> {'Alice': 30, 'Bob': 25, 'Mike': 43}
{'Alice': 30, 'Bob': 25, 'Mike': 43}

Literal syntax is particularly useful for singleton dictionaries:

q

q)enlist[`Alice]!enlist 30  / 4.0 syntax
Alice| 30
q)((), `Alice)!(), 30       / to save some typing
Alice| 30

q)([Alice:30])              / literal syntax
Alice| 30

Python

>>> {'Alice': 30}
{'Alice': 30}

Seeing keys and values next to each other is useful when creating larger dictionaries. Dictionary literal syntax works only with symbol keys, but you can use the over accumulator with ! for other types:

q

q)(!/) flip (("Alice"; 30); ("Bob"; 25); ("Mike"; 43))
"Alice"| 30
"Bob"  | 25
"Mike" | 43

Python

>>> dict([("Alice", 30), ("Bob", 25), ("Mike", 43)])
{'Alice': 30, 'Bob': 25, 'Mike': 43}

Empty dictionaries¶

You can create a general empty dictionary using general empty lists:

q)()!()     / general empty dictionary

In production, use typed dictionaries to prevent type promotion:

q)(`symbol$())!`float$()    / typed empty dictionary
q)([])~(`symbol$())!()      / empty dictionary with symbol keys

Projection from a dictionary literal syntax¶

Similar to list literal syntax (;..), omitting a value creates a projection:

q

q)d:([a:101; b:])    / missing values create projections
q)d 102
a| 101
b| 102

Python

>>> d = lambda x: {'a': 101, 'b':x}
>>> d(102)
{'a': 101, 'b': 102}

q)d each `AA`BB`CC  / create a list of dictionaries (table) via projection
a   b
------
101 AA
101 BB
101 CC

Implicit key names¶

Dictionary definition also makes it convenient to create a dictionary from variables.

q)Alice: 30
q)Bob: 25
q)([Alice; Bob])
Alice| 30
Bob  | 25

q assigns default key names when you omit them.

q)([0;1;2])
x | 0
x1| 1
x2| 2

Operations¶

Existence of a key¶

Use in to check the existence of a key.

q

q)d:([Alice: 30; Bob: 25; Mike: 43])
q)`Alice in key d
1b
q)`John in key d
0b

Python

>>> d = {'Alice': 30, 'Bob': 25, 'Mike': 43}
>>> 'Alice' in d
True
>>> 'John' in d
False
>>>

Lookup and indexing¶

To find the output value corresponding to an input key, you look up the key. Lookup uses the same notation as indexing into a list:

q

q)d:([Alice: 30; Bob: 25; Mike: 43])
q)d[`Bob]           / indexing
25
q)d `Bob            / brackets not necessary for indexing
25
q)d @ `Bob          / using 'index at'
25
q)d `Mike`Alice     / index by list
43 30
q)d `John           / indexing out of domain returns a null of the same type as the first value
0N

Python

>>> d = {'Alice': 30, 'Bob': 25, 'Mike': 43}
>>> d['Bob']
25




>>> [d[k] for k in ('Mike', 'Alice') if k in d]
[43, 30]

Indexed assignment¶

You can edit the value of any key:

q

q)d:([Alice: 30; Bob: 25; Mike: 43])
q)d[`Alice]: 40     / update
q)d
Alice| 40
Bob  | 25
Mike | 43
q)d[`John]: 30     / insert
q)d
Alice| 40
Bob  | 25
Mike | 43
John | 30
q)d[`Alice`Bob]: 50
q)d
Alice| 50
Bob  | 50
Mike | 43
John | 30
q)d[`Alice`Bob]: 60 70
q)d
Alice| 60
Bob  | 70
Mike | 43
John | 30

Python

>>> d = {'Alice': 30, 'Bob': 25, 'Mike': 43}
>>> d['Alice'] = 40
>>> d
{'Alice': 40, 'Bob': 25, 'Mike': 43}


>>> d.update(dict.fromkeys(['Alice', 'Bob'], 50))
>>> d
{'Alice': 50, 'Bob': 50, 'Mike': 43}


>>> d.update({'Alice': 60, 'Bob': 70})
>>> d
{'Alice': 60, 'Bob': 70, 'Mike': 43}

You can also use general amend at, if the dictionary name is provided as a symbol:

q)d:([Alice: 30; Bob: 25; Mike: 43])
q)@[`d;`Alice;:;50]
q)d
Alice| 50
Bob  | 25
Mike | 43

Subdictionaries¶

You can extract subdictionaries with the take operator. The inverse operator is cut:

q

q)d:([Alice: 30; Bob: 25; Mike: 43])
q)`Mike`Alice#d         / subdictionary
Mike | 43
Alice| 30
q)(enlist `Alice)#d
Alice| 30
q)`Mike`Alice cut d     / underscore is also accepted
Bob| 25

Python

>>> d = {'Alice': 30, 'Bob': 25, 'Mike': 43}
>>> {k: d[k] for k in ('Mike', 'Alice') if k in d}
{'Mike': 43, 'Alice': 30}
>>> {'Alice': d['Alice']}
{'Alice': 30}
>>> {k: v for k, v in d.items() if k not in ('Mike', 'Alice')}
{'Bob': 25}

Arithmetic operations¶

Dictionaries are a generalization of lists: the keys are predefined values rather than sequential integers. Most operators that work on lists also work on dictionaries.

Unary and binary arithmetic operations apply element-wise:

q)d:([Alice: 30; Bob: 25; Mike: 43])
q)2*d
Alice| 60
Bob  | 50
Mike | 86
q)d + ([Alice: 10; Bob: 5; John: 30])
Alice| 40
Bob  | 30
Mike | 43
John | 30
q)neg d
Alice| -30
Bob  | -25
Mike | -43
q)sums d
Alice| 30
Bob  | 55
Mike | 98
q)asc d
Bob  | 25
Alice| 30
Mike | 43
q)iasc d
`Bob`Alice`Mike
q)f: {2*x+1}
q)f d
Alice| 62
Bob  | 52
Mike | 88

Aggregations, some set functions, and some other functions only consider the values:

q

q)d:([Alice: 30; Bob: 25; Mike: 43])
q)count d
3
q)sum d
98
q)d except 25
30 43

Python

>>> d = {'Alice': 30, 'Bob': 25, 'Mike': 43}
>>> len(d)
3
>>> sum(d.values())
98
>>> [value for key, value in d.items() if value != 25]
[30, 43]

Recall that the keys are ordered, so you can reverse a dictionary, or take or drop its first or last elements:

q

q)reverse ([Alice: 30; Bob: 25; Mike: 43])
Mike | 43
Bob  | 25
Alice| 30
q)2 sublist ([Alice: 30; Bob: 25; Mike: 43])
Alice| 30
Bob  | 25
q)-2 _ ([Alice: 30; Bob: 25; Mike: 43])     / drop elements
Alice| 30

Python

>>> dict(reversed({'Alice': 30, 'Bob': 25, 'Mike': 43}.items()))
{'Mike': 43, 'Bob': 25, 'Alice': 30}


>>> dict(list({'Alice': 30, 'Bob': 25, 'Mike': 43}.items())[:2])
{'Alice': 30, 'Bob': 25}

>>> dict(list({'Alice': 30, 'Bob': 25, 'Mike': 43}.items())[:-2])
{'Alice': 30}

Iteration¶

Explicit iteration is rarer in q than in other languages, because many operators and functions already iterate for you. Where iteration does earn its place is with a function that is not atomic — one with a side effect, or one that needs to see a whole value at a time. each applies such a function to the values:

q

q)d:([Alice: 30; Bob: 25; Mike: 43])
q){show "the value is ", string x} each d;
"the value is 30"
"the value is 25"
"the value is 43"

Python

>>> d = {'Alice': 30, 'Bob': 25, 'Mike': 43}
>>> for v in d.values():
...     print("the value is", v)
the value is 30
the value is 25
the value is 43

Do not reach for each when the operator already iterates

An atomic function applies through a dictionary on its own, and doing it directly is both shorter and faster. Write 2*d, not {x*2} each d.

When the function is not atomic, each returns a dictionary with the keys preserved. count is a common case, since it must see each value whole:

q)show kids:([Alice: enlist `Bill; Bob: `$(); Mike: `Maggie`Jack`Ellie])
Alice| ,`Bill
Bob  | `symbol$()
Mike | `Maggie`Jack`Ellie
q)count each kids
Alice| 1
Bob  | 0
Mike | 3

Iterate over one side on its own by taking it as a plain list with key or value. The result is then a list, not a dictionary:

q

q)key d
`Alice`Bob`Mike
q)value d
30 25 43
q){x+1} each value d
31 26 44

Python

>>> list(d.keys())
['Alice', 'Bob', 'Mike']
>>> list(d.values())
[30, 25, 43]
>>> [v+1 for v in d.values()]
[31, 26, 44]

To visit each key together with its value, apply a binary function with the each-both iterator ('):

q

q){string[x], ": ", string y}'[key d; value d] / or (key d) {...}' value d
"Alice: 30"
"Bob: 25"
"Mike: 43"

Python

>>> [f"{k}: {v}" for k, v in d.items()]
['Alice: 30', 'Bob: 25', 'Mike: 43']

Filtering a dictionary is a lookup rather than a loop. where returns the keys that satisfy a condition, and taking those keys with # gives back a dictionary:

q

q)where d > 28
`Alice`Mike
q)(where d > 28) # d
Alice| 30
Mike | 43

Python

>>> [k for k, v in d.items() if v > 28]
['Alice', 'Mike']
>>> {k: v for k, v in d.items() if v > 28}
{'Alice': 30, 'Mike': 43}

Reverse lookup¶

Use find for a reverse lookup on a dictionary:

q

q)d:([Alice: 30; Bob: 25; Mike: 43; John: 30])
q)d?25          / find
`Bob
q)d?40          / value not found works as for lists
`               / returns a null of the same type as the first key
q)d?30          / only returns the first match
`Alice

Python

>>> d = {'Alice': 30, 'Bob': 25, 'Mike': 43, 'John': 30}
>>> next(k for k, v in d.items() if v == 25)
'Bob'


>>> next(k for k, v in d.items() if v == 30)
'Alice'

You can also use vector operation = and keyword where:

q

q)where d=25
,`Bob
q)where d=40
`symbol$()
q)where d=30
`Alice`John

Python

>>> [k for k, v in d.items() if v == 25]
['Bob']
>>> [k for k, v in d.items() if v == 40]
[]
>>> [k for k, v in d.items() if v == 30]
['Alice', 'John']

Joins¶

The Join operator (,) merges two dictionaries.

Join on dictionaries has upsert semantics for common keys. If a key exists in both dictionaries, the value from the right operand overwrites the value from the left operand. New keys are appended to the end.

Example:

q

q)([Alice: 30; Bob: 25; Mike: 43]), ([Alice: 10; Bob: 5; John: 30])
Alice| 10
Bob  | 5
Mike | 43
John | 30

Python

>>> {'Alice': 30, 'Bob': 25, 'Mike': 43} | {'Alice': 10, 'Bob': 5, 'John': 30}
{'Alice': 10, 'Bob': 5, 'Mike': 43, 'John': 30}

Null replacement with Coalesce (`^`)¶

The coalesce operator ^ uses upsert semantics to merge two dictionaries, with the right operand prevailing over the left on common keys. The difference from , is that null values in the right operand do not prevail over the left.

q)left: ([Alice: 30; Bob: 0N; Mike: 43])
q)right: ([Alice: 10; Bob: 5; Mike: 0N])
q)left ^ right
Alice| 10
Bob  | 5
Mike | 43

Column dictionaries¶

A dictionary whose value items are all same-length lists is a column dictionary. Column dictionaries are the foundation for tables.

Example:

q)show friends:([name:`Alice`Bob`Mike; dob: 1982.09.15 1984.07.05 1990.11.16; sex: "mfm"])
name| Alice      Bob        Mike
dob | 1982.09.15 1984.07.05 1990.11.16
sex | m          f          m
q)count each friends
name| 3
dob | 3
sex | 3

A column dictionary looks like a matrix, but you index by a key:

q)friends[`name; 1]
`Bob
q)friends . (`name;1)
`Bob

Flip a column dictionary and the result is a table:

q)flip friends
name  dob        sex
--------------------
Alice 1982.09.15 m
Bob   1984.07.05 f
Mike  1990.11.16 m

Step dictionaries¶

A step dictionary is a dictionary with the sorted attribute applied. When a key is not present, it returns the value associated with the nearest preceding key instead of a null value.

Note

The sorted attribute needs to be applied to the keys of the dictionary, as well as to the dictionary as a whole for it to function as a step dictionary. If the keys are unsorted, q throws 's-fail when you try to create a step dictionary.

An as-of join is an example of a step dictionary: it joins the prevailing value to the time field in the join.

Example:

q)d:(00:00:00; 04:00:00; 09:00:00)!`closed`preopen`open
q)ds:`s#d
q)d 06:00:00
`
q)ds 06:00:00
`preopen

Next steps¶

See Q for Mortals §5. Dictionaries for additional detail on dictionaries.
See tables, which are lists of dictionaries.

How to work with dictionaries¶

Dictionary creation¶

Dictionary literal syntax¶

Empty dictionaries¶

Projection from a dictionary literal syntax¶

Implicit key names¶

Operations¶

Existence of a key¶

Lookup and indexing¶

Indexed assignment¶

Subdictionaries¶

Arithmetic operations¶

Iteration¶

Reverse lookup¶

Joins¶

Null replacement with Coalesce (^)¶

Column dictionaries¶

Step dictionaries¶

Next steps¶

Null replacement with Coalesce (`^`)¶