each, peach¶
Iterate a unary
v1 each x each[v1;x] v1 peach x peach[v1;x]
(vv)each x each[vv;x] (vv)peach x peach[vv;x]
Where
v1is a unary applicable valuevvis a variadic applicable value
applies v1 or vv as a unary to each item of x and returns a result of the same length.
That is, the projections each[v1;], each[vv;], peach[v1;], and peach[vv;] are uniform functions.
q)count each ("the";"quick";"brown";"fox")
3 5 5 3
q)(+\)peach(2 3 4;(5 6;7 8);9 10 11 12)
2 5 9
(5 6;12 14)
9 19 30 42
each and peach perform the same computation and return the same result.
peach divides the work between the available secondary threads. If q runs without secondary threads, peach is the same as each. For details, refer to Parallel processing.
each is a wrapper for the Each iterator.
peach is a wrapper for the Each Parallel iterator.
It is good q style to use each and peach for unary values.
each is redundant with atomic functions
Higher-rank values¶
peach applies only unary values.
For a value of rank 2 or higher, use Apply to project it as a unary value.
For example, suppose m is a 4-column matrix and each row has values for the arguments of v4. Then .[v4;]peach m applies v4 to each list of arguments.
q)v4:{[a;b;c;d]a+b+c+d}
q)m:(1 2 3 4;5 6 7 8)
q).[v4;]peach m
10 26
Alternatively, suppose t is a table in which columns b, c, and a are arguments of v3. Then .[v3;]peach flip t `b`c`a applies v3 to the arguments in each row of t.
Memory in secondary threads¶
Each secondary thread allocates from its own heap, which has two consequences.
A peach can use far more memory than the equivalent each. Where a serial iteration reuses one working set, a parallel one holds as many at once as there are threads. Here f allocates a temporary the size of its argument, so 8 threads hold 8 temporaries at once:
q)v:8#enlist 2000000?100.0 / 8 x 16MB vectors
q)f:{sum x*x}
Neither \ts nor .Q.w accounts for it, because both report the calling thread only. With 8 secondary threads (q -s 8), the parallel version appears to allocate almost nothing:
q)system"ts f each v"
3 16778384
q)system"ts f peach v"
3 1648
The figures vary slightly between runs.
The session reports 16 MB for each against 1.6 KB for peach. Measured externally, however, the process running peach peaked at around 159 MB against 47 MB for each. The difference is the per-thread temporaries that \ts cannot see. So a peach that reports no allocation might still be the reason a process runs out of memory.
If the footprint matters, measure the process rather than the session: /usr/bin/time -l on macOS, /usr/bin/time -v on Linux, or the resident size reported by the operating system while it runs.
Blocked within peach¶
peach blocks the following:
hopenon a socket- opening a WebSocket
- socket broadcast (
25!x) - amending global variables
- loading the master decryption key (
-36!) - any system command that might change global state
These restrictions apply when q runs with secondary threads. Without them, peach runs in the main thread and nothing is blocked.
q)/ q started with -s 2
q){a::x} peach 1 2
'noupdate: `. `a
[0] {a::x} peach 1 2
^
[2] {a::x}
^
Generally, do not use a socket within peach unless it is inside a one-shot sync request or an HTTP client request. peach blocks any other socket use and signals a nosocket error.
You can open and close files within peach, provided no two threads use the same file handle or file at the same time.
You can also use streaming execute (-11!) within peach. However, because peach blocks updates to global variables, it is useful in fewer cases.
Maps for uses of Each with binary and higher-rank values
Table counts in a partitioned database
Q for Mortals
A.49 peach