Skip to content

each, peach

Iterate a unary

 v1 each x   each[v1;x]       v1 peach x   peach[v1;x]  
(vv)each x   each[vv;x]      (vv)peach x   peach[vv;x]

Where

applies v1 or vv as a unary to each item of x and returns a result of the same length.

That is, the projections each[v1;], each[vv;], peach[v1;], and peach[vv;] are uniform functions.

q)count each ("the";"quick";" brown";"fox")
3 5 6 3
q)(+\)peach(2 3 4;(5 6;7 8);9 10 11 12)
2 5 9
(5 6;12 14)
9 19 30 42

each and peach perform the same computation and return the same result.

peach will divide the work between available secondary tasks. Refer to Parallel processing for details.

each is a wrapper for the Each iterator. peach is a wrapper for the Each Parallel iterator. It is good q style to use each and peach for unary values.

each is redundant with atomic functions.

Maps for uses of Each with binary and higher-rank values

.Q.fc parallel on cut

Parallel processing

Table counts in a partitioned database

Q for Mortals A.49 peach

Higher-rank values

peach applies only unary values. For a values of rank ≥2, use Apply to project v as a unary value.

For example, suppose m is a 4-column matrix and each row has values for the arguments of v4. Then .[v4;]peach m will apply v4 to each list of arguments.

Alternatively, suppose t is a table in which columns b, c, and a are arguments of v3. Then .[v3;]peach flip t `b`c`a will apply v3 to the arguments in each row of t.

Memory in secondary threads

Each secondary thread allocates from a heap of its own, and this has two consequences that surprise people.

A peach can use far more memory than the equivalent each. Where a serial iteration reuses one working set, a parallel one holds as many at once as there are threads. Here f allocates a temporary the size of its argument, so eight threads hold eight of them:

q)v:8#enlist 2000000?100.0      / 8 x 16MB vectors
q)f:{sum x*x}

Neither \ts nor .Q.w accounts for it, because both report the calling thread only. The parallel version appears to allocate almost nothing:

q)system"ts f each v"
3 16778768
q)system"ts f peach v"
3 1648

16MB against 1.6KB — yet measured externally, the process running peach peaked around 158MB against 44MB for each, the extra being the per-thread temporaries that \ts cannot see. So a peach that reports no allocation may still be the reason a process runs out of memory.

If the footprint matters, measure the process rather than the session: /usr/bin/time -l on macOS, /usr/bin/time -v on Linux, or the resident size reported by the operating system while it runs.

Blocked within peach

hopen socket
websocket open
socket broadcast (25!x)
amending global variables
load master decryption key (-36!)

And any system command which might cause a change of global state.

Generally, do not use a socket within peach, unless it is encapsulated via one-shot sync request or HTTP client request (TLS/SSL support added in 4.1t 2023.11.10). Erroneous socket usage is blocked and signals a nosocket error.

If you are careful to manage your file handles/file access so that there is no parallel use of the same handle (or file) across threads, then you can open and close files within peach.

Streaming execute (-11!) should also be fine. However updates to global variables are not possible, so use cases might be quite restricted within peach.