1. Shock and Awe

1.1 Starting Q

Your installation of q should have placed the q executable in $HOME/q (or $QHOME) on Unix-based systems, or in the q directory on the c: drive on Windows.

For Windows, start a q session by typing q on the command line; for Linux-based systems use rlwrap q so that you will have command line recall. You should see a new q session with the Kx Systems copyright notice followed by the q prompt indicated by a leading q) on the command line. This is the q console. Type 6*7 and press Enter or Return to see the result.

q)6*7
42
q)☐

Here the ☐ represents the cursor awaiting your next input.

Tip

In this document, sample q console sessions will always be displayed in fixed pitch font with shaded background. We expect you, the serious student, to enter this expression into your own your q session and see the result. Otherwise all you will learn is that the author knows q.

q)"c"$0x57656c6c20646f6e6521
_

You did do this, didn't you? This is a tutorial.

1.2 Variables

A variable in q is a name together with associated storage that holds whatever value has been most recently assigned to the variable.

Tip

In q, as in most languages that allow mutable state, a "variable" should properly be called an "assignable". (See Mathematical Refresher above for a discussion). As a consolation, at least q does not abuse = for assignment, as do many languages; in q = actually means "test for equality."

Declaring a variable and assigning its value are done in a single step with the operator : which is called "amend" and is read "is assigned" or "gets." Here is how to create and assign variable a with integer value 42.

q)a:42
q)_

When you entered this in your q session, you will have noted that nothing is echoed to the console. In order to see that a has indeed been assigned, simply enter the variable.

q)a
42

A variable name must start with an alphabetic character, which can be followed by alpha, numeric or underscore.

Naming Style Recommendations

Choose a name long enough to make the purpose of the entity evident, but no longer. The purpose of a name is to communicate to a reader of the code at another time - perhaps even you. Long names may not make code easier to read. For example, checkDisk is clearer than cd or chk but is no less clear than checkDiskForFileConsistency in most contexts.
Use verbs for function names; use nouns for data.
Be consistent in your use of abbreviations. Be mindful that even "obvious" abbreviations may be opaque to readers whose native language is different than yours.
Be consistent in your use of capitalization, such as initial caps, camel casing, etc. Pick a style and stick to it.
Use contexts to place functions in namespaces.
Do not use names such as int, float or other words that have meaning in q. While not reserved, they carry special meaning when used as arguments for certain q operators.
Accomplished q programmers avoid using the underscore character in q names. Aside from making your q code look like verbose Java or Python, if you insist on using underscore in names, do not use it as the last character. While this is permitted, expressions involving the built-in _ operator and names with underscore will be difficult to read.

1.3 Whitespace

In general, q permits, but does not require, whitespace around operators, separators, brackets, braces, etc. You could also write the above expression as

a : 42

or

a: 42

Tip

Accomplished q programmers view whitespace around operators as training wheels on a bicycle. They may be polite to your face but they will snicker.

Because the q gods prefer compact code, you will see programs with no superfluous whitespace...none, zilch, zip, nada. In order to help you get accustomed to this terseness, we use whitespace only in juxtaposition and after semicolon and comma separators. You should feel free to add whitespace for readability where it is permitted, but be consistent in its use or omission. We will point out where whitespace is required or forbidden.

1.4 The Q Console

The q console evaluates a q expression that you enter and echoes the result on the following line. An exception to this is the assignment operation - as noted above - that has a return value, even though the console does not echo it. You may wonder why. This is simply a q console design choice to avoid cluttering the display.

Note

To obtain the normal console display of any q value, apply the built-in function (show) to it.

q)show a:42
42

1.5 Comments

The forward-slash character / indicates the beginning of a comment. Actually, it instructs the interpreter to ignore everything from it to the end of the line.

Tip

At least one whitespace character must separate / intended to begin a comment from any text to the left of it on a line.

In the following example, no definition of c is seen by the interpreter so an error occurs.

q)b:1+a: 42 / nothing here counts     c:6*7
q)c
'c
  [0]  c
       ^

Notice the succinct (ahem) format of a q error message: a single vertical quote followed by a terse error message. In this case the message is telling you that q didn't recognize c and is pointing you to level 0 of the input stack where it was encountered. This display is more detailed than earlier versions of q that simply echoed 'c.

The following generates an even more succinct error.

q)a:42/ intended to be a comment
'
[0] a:42/ intended to be a comment

Coding Style Recommendation

The q gods have no need for explanatory error messages or comments since their q code is perfect and self-documenting. Even experienced mortals pore over cryptic q error messages such as the ones above. Moreover, many mortals eschew comments in misanthropic coding macho. Just don't.

1.6 Assignment

A variable is not explicitly declared or typed. Instead, its assigned value determines the variable's type. In our example, the expression to the right of the assignment is syntactically an integer value, so the name a is associated with a value of type long integer. It is permissible to reassign a variable with a value of different type. Once this is done, the name will reflect the type of its newly assigned value. Much more about types in Chapter 2.

Important

Dynamic typing combined with mutable variables is flexible but also dangerous. You can unintentionally change the type of a variable with a wayward assignment that might cause your program to terminate unexpectedly much later. Or you can inadvertently reuse a variable name and wipe out any data in the variable. An undetected typo can result in data being sent to a black hole. Be careful to enter variable names correctly especially if you follow the q school that makes variable names short and cryptic.

Many traditional languages permit only a variable name to the left of an assignment. In q an assignment carries the value being assigned and can be used as part of a larger expression. So we find,

q)1+a:42
43

In the following example, the variable a is not referenced after it is assigned. Instead, the value of the assignment is propagated onward - i.e., to the left.

q)b:1+a:42
q)b
43

Tip

To see all the variables in the current workspace issue the command \v. The funky `s# means the list is sorted. In a fresh q session,

q)b:1+a:24
q)\v
`s#`a`b

1.7 Order of Evaluation

The interpreter evaluates the above specification of b from right-to-left (more on this in Chapter 4). If it were verbose, the interpreter might say:

The integer 42 is assigned to a variable named a, then the result of the assignment, namely 42, is added to the integer 1, then this result is assigned to a variable named b

Because the interpreter always evaluates expressions right-to-left, programmers can safely read q expressions left-to-right,

The variable b gets the value of the integer 1 plus the value assigned to the variable a, which gets the integer 42

This is exactly as in mathematics where we would read f(g(x)) as "f of g of x" even though g is evaluated first and the result passed into f. We just dispense with the parentheses.

Recommendations on Assignment Style

The ability to chain evaluation of expressions permits a single line of q code to perform the work of an entire verbose program. In general this is acceptable (even good) q style when not taken to the extreme with long wrapped lines or nested sub expressions containing assignments.
Intra-line assignments, as above, can simplify code provided they are few and are referenced only within the line of creation.
It is not bad form to make one assignment per line, provided you don't end up with one operation per line.
Wannabe q gods carry terseness to the extreme, which quickly leads to write-only code.

1.8 Data Types 101

There are q data types to cover nearly all needs but a few basic types are used most frequently. In q the basic integer type, called long, is a 64 bit signed integer. If you write a literal integer as in the snippets above, q creates a 64 bit signed integer value.

q)42
42

The basic floating point type in q is called float, often called "double" in many languages. This is an 8 byte value conforming to the IEEE floating point specification.

q)98.6
98.6

Arithmetic operations on integer and float values are mostly as expected except for division, which is written as % since / has been preempted for comments and other uses. Sorry, that's just the way it is. Also note that division always results in a float.

q)2+3
5
q)2.2*3.0
6.6
q)4-2
2
q)4%2
2f

Boolean values in q are stored in a single byte and are denoted as the binary values they really are with an explicit type suffix b. One way to generate boolean values is to test for equality.

q)42=40+2
1b
q)42=43
42=43

The two most useful temporal types are date and timespan; both represent integral counts. Under the covers, a date is the number of days since the millennium, positive for post and negative for pre.

q)2000.01.01    / this is actually 0
2000.01.01
q)2024.11.19    / this is actually 5436
2024.11.19
q)1999.12.31    / this is actually -1
1999.12.31

Similarly, a time value is represented by a timespan, which is a long integer count of the number of nanoseconds since midnight. It is denoted as,

q)12:00:00.000000000    / this is noon
0D12:00:00.000000000

One interesting and useful feature of q temporal values is that, as integral values under the covers, they naturally participate in arithmetic. For example, to advance a date five days add 5.

q)2000.01.01+5
2000.01.06

Or to advance a time by one microsecond (i.e., 1000 nanoseconds) add 1000.

q)12:00:00.000000000+1000
0D12:00:00.000001000

Or to verify that temporal values are indeed their underlying values, test for equality.

q)2000.01.01=0
1b
q)12:00:00.000000000=12*60*60*1000000000
1b

The treatment of character data in q is a bit complicated in order to provide optimum flexibility and performance. For now, we will focus on symbols, which derive from their namesake in Scheme and are akin to VARCHAR in SQL or strings in other languages. They are not what q calls strings!

Think of symbols as wannabe names: all q names are symbols but not all symbols are names. A symbol is atomic, meaning that it is viewed as an indivisible entity (although we shall see later how to expose the characters inside it).

Symbols are denoted by a leading backtick followed by characters. Symbols without embedded blanks or other special characters can be entered literally into the console,

q)`aapl
`aapl
q)`jab
`jab
q)`thisisareallylongsymbol
`thisisareallylongsymbol

Since symbols are atoms, they can be tested for equality.

q)`aapl=`apl
0b

1.9 Lists 101

The fundamental q data structure is a list, which is an ordered collection of items sequenced from left to right. The notation for a general list encloses items with ( and ) and uses ; as separator. Spaces are optional.

q)(1; 1.2; `one)
1
1.2
`one

A few observations on lists.

A list can contain items of different types; this usually requires wrapping the values in a variant type or boxing in other languages. That being said, it is best to avoid mixed types in lists when possible, as their processing can be slower than homogeneous lists of atoms. Recent versions of kdb have narrowed the difference significantly.
Assuming you entered the above snippet, you have noticed that the console echoes the items of a general list one item per line.
In contrast to most functional languages, lists need not be built up by "cons"-ing one item at a time, although they can be. Nor are they stored as singly-linked lists under the covers.

In the case of a homogeneous list of atoms, called a simple list - also by abuse of terminology a vector - q adopts a simplified format for both storage and display. The parentheses and semicolons are dropped. For example, a list of underlying numeric type separates its items with a space.

q)(1; 2; 3)
1 2 3
q)(1.2; 2.2; 3.3)
1.2 2.2 3.3
q)(2000.01.01; 2000.01.02; 2001.01.03)
2000.01.01 2000.01.02 2001.01.03

A simple list of booleans is juxtaposed with no spaces and has the trailing type indicator.

q)(1b; 0b; 1b)
101b

A simple list of symbols is displayed with no separating spaces.

q)(`one; `two; `three)
`one`two`three

Homogeneous lists of atoms can be entered in either general or simplified form. Regardless of how they are created, q recognizes a list of homogeneous atoms dynamically and converts it to a simple list.

Next we explore some basic operations to construct and manipulate lists. The most fundamental is til, which takes a non-negative integer n and returns the first n integers starting at 0 (n itself is not included in the result).

q)til 10
0 1 2 3 4 5 6 7 8 9

We obtain the first 10 integers starting at 1 by adding 1 to the previous result. Be mindful that q always evaluates expressions from right-to-left and that operations work on vectors whenever possible.

q)1+til 10
1 2 3 4 5 6 7 8 9 10

Similarly, we obtain the first 10 even numbers and the first ten odd numbers.

q)2*til 10
0 2 4 6 8 10 12 14 16 18
q)1+2*til 10
1 3 5 7 9 11 13 15 17 19

Another frequently used list primitive is Join , which returns the list obtained by concatenating its right operand to its left operand.

q)1 2 3,4 5
1 2 3 4 5
q)1 2 3,100
1 2 3 100
q)0,1 2 3
0 1 2 3

To extract items from the front or back of a list, use the Take operator #. Positive argument means take from the front, negative from the back.

q)2#til 10
0 1
q)-2#til 10
8 9

Should you attempt to extract more items that there are in the list, # returns to the beginning and restarts extracting. It does this until the specified number of items is reached.

q)5#1 2 3
1 2 3 1 2

In particular, if you apply # to an atom, it will continue drawing that single atom until it has the specified number of copies. In other words, it replicates an atom to a list.

q)5#42
42 42 42 42 42

As with atoms, a list can be assigned to a variable.

q)L:10 20 30

The items of a list can be accessed via indexing, which uses square brackets and is relative to 0.

q)L[0]
10
q)L[1]
20
q)L[2]
30

1.10 Functions 101

All built-in q operators are functions. The main differences between q's functions and the ones we mortals can write are:

The built-ins are written and optimized in one of the underlying languages k or C.
All q functions can be used with infix notation - i.e., as operators - whereas ours must be used in prefix form.

Functions in q correspond to "lambda expressions" or "anonymous functions" in other languages. This means that a function is a first class value just like a long or float value - i.e., it only acquires a name once it is assigned to a variable.

Conceptually, a q function is a sequence of steps that produces an output result from an input value. Since q is not purely functional, these rules can interact with the world by reaching outside the internal context of the function. Such actions are called side effects and should be avoided when possible and carefully controlled when not.

Function definition is delimited by matching curly braces { and }. Immediately after the opening brace, the formal parameters are names enclosed in square brackets [ and ] and separated by semicolons. These parameters presumably appear in the body of the function, which follows the enclosed formal parameters and is a succession of expressions sequenced (not terminated) by semicolons.

Following is a simple function that returns the square of its input. On the next line we assign the same function to the variable sq. The whitespace is optional.

q){[x] x*x}
{[x] x*x}
q)sq:{[x] x*x}
q)sq
{[x] x*x}

Here is a function that takes two input values and returns the sum of their squares.

q){[x;y] a:x*x; b:y*y; a+b}
{[x;y] a:x*x; b:y*y; a+b}
q)pyth:{[x;y] a:x*x; b:y*y; a+b}

To apply a function to arguments, follow it (or its name, if it has been assigned to a variable) by a list of values enclosed in square brackets and separated by semicolons. This causes any argument expression to be evaluated first, then the expressions in the body of the function to be evaluated sequentially by substituting each resulting argument for every occurrence of the corresponding formal parameter. Normally the value of the final expression is returned as the output value of the function.

Here are the previous functions applied to arguments.

q){[x] x*x}[5]
25
q)sq[5]
25
q){[x;y] a:x*x; b:y*y; a+b}[3;4]
25
q)pyth[3;4]
25

The variables a and b appearing in the body of the last function above are local - i.e., they are created and exist only for the duration of an application.

It is common in mathematics to use function parameters x, y, or z. If you are content with these names (in the belief that descriptive names provide no useful information to the poor soul reading your code), you can omit their declaration and q will understand that you mean x, y, and z in that order.

q){x*x}[5]
25
q){a:x*x; b:y*y; a+b}[3;4]
25

You might think this is about as pithy as it gets for function definition and application. Well, not quite. In q, as in most functional languages, we don't need no stinkin' brackets for application of a unary function - i.e., with one parameter. This means that you can separate the function name from its argument by whitespace. This is called function juxtaposition.

q){x*x} 5         / white space optional here   
25
q)f:{x*x}
q)f 5              / white space required here  
25

If you are new to functional programming this may take some getting used to, but the reduction of code "noise" is worth it.

1.11 Functions on Lists 101

Because q is a vector language, most of the built-in operators work on lists out of the box. In q-speak, such functions are atomic, meaning they recursively burrow into a complex data structure until arriving at atoms and then perform their operation. In particular, an atomic function operates on lists by application to the individual items. For example, plain addition adds an atom to a list, a list to an atom or two lists of the same length.

q)42+100 200 300
142 242 342
q)100 200 300+42
142 242 342
q)100 200 300+1 2 3
101 202 303

Perhaps surprisingly, this is also true of equality and comparison operators. (Recall the notation for simple boolean lists).

q)100=99 100 101
010b
q)100 100 100=100 101 102
100b
q)100<99 100 101
001b

Suppose that instead of adding things pair-wise, we want to add all the items across a list. The way this is done in functional languages is with higher order functions, or as they were formerly called in q, adverbs. Regardless of the terminology, the idea is to take the operation of a function and produce a closely related function having the same "essence" but applied in a different context.

You met the concept of higher-order functions in elementary calculus, perhaps without being properly introduced. The derivative and integral are actually higher order functions that take a function and produce a related function. Behind all the delta-epsilon mumbo-jumbo, the derivative of a given function is a function that represents the instantaneous behavior of the original. The (indefinite) integral is the anti-derivative - i.e., a function whose instantaneous behavior is that of the given function.

In the case of adding the values in a list, we need a higher-order function that takes addition and turns it into a function that works across the list. In functional programming this is called a fold; in q it is "over." The technique is to accumulate the result across the list recursively. (See Mathematical Refresher for more on recursion). Specifically, begin with an initial value in the accumulator and then sequentially add each list item into the previous value of the accumulator until the end of the list. Upon completion, the accumulator holds the desired result.

If you are new to functional programming this may seem more complicated than just creating a for loop but that's only because you have been brainwashed. Watch how easy it is to do in q. In words, we tell q to start with the initial value of 0 in the accumulator and then modify + with the iterator / so that it adds across the list.

q)0 +/ 1 2 3 4 5         / whitespace optional
15
q)0 +/ 1+til 100
5050

There is nothing special about the built-in operator + - we can use any operator or even our own function.

q)0 {x+y}/ 1 2 3 4 5
15
q)0 {x+y}/ 1+til 100
5050

In this situation we don't really need the flexibility to specify the initial value of the accumulator. It suffices to start with the first item of the list and proceed across the rest of the list. There is an even simpler form for this case.

q)(+/) 1 2 3 4 5
15
q)(+/) 1+til 100
5050

If you are new to functional programming, you may think, "Big deal, I write for loops in my sleep." Granted. But the advantage of the higher-order function approach is that there is no chance of being off by one in the loop counter or accidentally running off the end of a data structure. More importantly, you can focus on what you want done without the irrelevant scaffolding of how to set up control structures. This is called declarative programming. It must be noted that many imperative languages have added for-each constructs since this text was originally written.

What else can we do with our newfound iterator? Change addition to multiplication for factorial.

q)(*/) 1+til 10
3628800

The fun isn't limited to arithmetic primitives. We introduce |, which returns the larger of its operands and &, which returns the smaller of its operands.

q)42|98
98
q)42&98
42

Use | or & with over and you have maximum or minimum.

q)(|/) 20 10 40 30
40
q)(&/) 20 10 40 30
10

Some applications of / are so common that they have their own names.

q)sum 1+til 10          / this is +/
55
q)prd 1+til 10         / this is */  -- note missing “o”
3628800
q)max 20 10 40 30       / this is |/
40
q)min 20 10 40 30      / this is &/
10

At this point the / pattern should be clear: it takes a given function and produces a new function that runs the original across a list, producing a single result. In particular, / converts a binary function to an aggregate function - i.e., one that collapses a list to an atom.

We record one more example of / for later reference. Recall from the previous section that applying the operator # to an atom produces a list of copies. Composing this with */ we get a multiplicative implementation of raising to a power without resorting to the floating point exponential operator.

q)(*/) 2#1.4142135623730949
1.9999999999999996
q)n:5
q)(*/) n#10
100000

The higher-order function sibling to over is scan, written \. The process of scan is the same as that of over with one difference: instead of returning only the final result of the accumulator, it returns all intermediate values.

q)(+\) 1+til 10
1 3 6 10 15 21 28 36 45 55
q)(*\) 1+til 10
1 2 6 24 120 720 5040 40320 362880 3628800
q)(|\) 20 10 40 30
20 20 40 40
q)(&\) 20 10 40 30
20 10 10 10

Scan converts a binary function to a uniform function - i.e., one that returns a list of the same length as the input.

As with over, common applications of scan have their own names.

q)sums 1+til 10             / this is +\
1 3 6 10 15 21 28 36 45 55
q)prds 1+til 10              / this is *\   -- note missing “o”
1 2 6 24 120 720 5040 40320 362880 3628800
)maxs 20 10 40 30           / this is |\
20 20 40 40
q)mins 20 10 40 30          / this is &\
20 10 10 10

1.12 Example: Fibonacci Numbers

We define the Fibonacci numbers recursively.

Base case: the initial sequence is the list 1 1
Inductive step: given a list of Fibonacci numbers, the next value of the sequence appends the sum of its two last items.

We have the basic ingredients to express this in q. Start with the base case F0.

q)F0:1 1
q)-2#F0
1 1
q)sum -2#F0
2
q)F0,sum -2#F0
1 1 2

Notice that read from right-to-left, the last expression exactly restates the definition of the Fibonacci term: "take the last two elements of the sequence, sum them and append the result to the sequence." This is declarative programming - say "what" to do not "how" to implement it.

We abstract this expression into a function that appends the next item at an arbitrary point in the sequence.

q){x,sum -2#x}

Let's take it for a test drive on the first few terms.

q){x,sum -2#x}[1 1]
1 1 2
q){x,sum -2#x}[1 1 2]
1 1 2 3

Wouldn't it be nice if q had a higher-order function that applies a recursive function a specified number of times, starting with the base case? Conveniently, there is an overload of our friend / that does exactly this. Specify the base case and the number of times to iterate the recursion and it's done.

q)10 {x,sum -2#x}/ 1 1
1 1 2 3 5 8 13 21 34 55 89 144

1.13 Example: Newton's Method for n^th Roots

You may recall from elementary calculus the simple and powerful technique for computing roots of functions, called the Newton-Raphson method. (For those brave enough to pursue Newton's original, this bears little superficial resemblance to what Newton himself originally wrote). The idea is to start with an initial guess that is not too far from the actual root. Then determine the tangent to the graph over that point and project the tangent line to the x-axis to obtain the next approximation. Repeat this process until the result converges within the desired tolerance.

We formulate this as a recursive algorithm for successive approximation.

Base case: a reasonable initial value
Inductive step: Given x_n, the n+1^st approximation is: x_n -- f(x_n) / f'(x_n)

Let's use this procedure to compute the square root of 2. The function whose zero we need to find is f(x) = x² - 2. The formula for successive approximation involves the derivative of f, which is f'(x) = 2x*.

Given that we know that there is a square root of 2 between 1 and 2 due to the sign change of f, we start with 1.0 as the base case x0. Then the first approximation is,

q)x0-((x0*x0)-2)%2*x0
1.5

We abstract this expression to a function that computes the n+1^st approximation in terms of xn

q){[xn] xn-((xn*xn)-2)%2*xn}
_

Now use it to run the first two iterations. 

q){[xn] xn-((xn*xn)-2)%2*xn}[1.0]
1.5
q){[xn] xn-((xn*xn)-2)%2*xn}[1.5]
1.416667

Observe in your console session that this looks promising for convergence to the correct answer.

Wouldn't it be nice of q had a higher-order function to apply a function recursively, starting at the base case, until the output converges? You won't be surprised that there is another overload of our friend over that does exactly this. Just specify the base case and q iterates until the result converges within its system comparison tolerance (as of this writing, that tolerance is 10^-14^)

q){[xn] xn-((xn*xn)-2)%2*xn}/[1.5]
1.414214

To witness the convergence, do two things. First, set the floating point display to maximum.

q)\P 0  / note upper case

Tip

This displays all digits of the underlying binary representation, including the 17^th digit, which is usually schmutz.

Second, switch the iterator from Over to Scan so that we can see the intermediate results.

q){[xn] xn-((xn*xn)-2)%2*xn}\[1.0]
1 1.5 1.4166666666666667 1.4142156862745099 1.4142135623746899
1.4142135623730951

As the console display shows, that is pretty fast convergence.

Why limit ourselves to the square root of 2? Abstracting the constant 2 into a parameter c in the function f, the successive approximation function becomes,

q){[c; xn] xn-((xn*xn)-c)%2*xn}

At this point we use a feature, related to currying in functional programming and called projection in q, in which we only partially supply arguments to a function. The result is a function of the remaining, un-specified parameters. We indicate partial application by omitting the unspecified arguments. In our case, we specify the constant c as 2.0, leaving a unary function of the remaining variable xn.

q){[c; xn] xn-((xn*xn)-c)%2*xn}[2.0;]

Since this is solely a function of xn, we can apply it recursively to the base case until it converges to obtain the same result as the original square root function.

q){[c; xn] xn-((xn*xn)-c)%2*xn}[2.0;]/[1.0]
1.4142135623730951

But now we are free to choose any (reasonable) value for c. For example, to calculate the square root of 3.0.

q){[c; xn] xn-((xn*xn)-c)%2*xn}[3.0;]/[1.0]
1.7320508075688772

Intoxicated with the power of function abstraction and recursion, why restrict ourselves to square roots? We abstract once more, turning the power into a parameter p. The new expression for the successive approximation has a p^th power in the numerator and an p-1^st power in the denominator, but we already know how to calculate these.

q){[p; c; xn] xn-(((*/)p#xn)-c)%p*(*/)(p-1)#xn}

Supplying only p and c leaves a function solely of xn, which we can once again iterate on the base case until convergence. We reproduce the previous case of the square root of 3.0; then we calculate the fifth root of 7.

q){[p; c; xn] xn-(((*/)p#xn)-c)%p*(*/)(p-1)#xn}[2; 3.0;]/[1.0]
1.7320508075688772
q){[p; c; xn] xn-(((*/)p#xn)-c)%p*(*/)(p-1)#xn}[5; 7.0;]/[1.0]
1.4757731615945617

It is amazing what can be done in a single line of code when you strip out unnecessary programming frou-frou. Perhaps this is intimidating to the qbie, but now that you have taken the blue pill, you'll feel right as rain.

1.14 Example: FIFO Allocation

In the Finance industry, one needs to fill a sell order from a list of matching buys in a FIFO fashion. Although we state this scenario in terms of buys and sells, it applies equally to general FIFO allocation. We begin with the buys represented as a (time-ordered) list of floats, and a single float sell.

q)buys:2 1 4 3 5 4f
q)sell:12f

The objective is to draw successively from the buys until we have exactly filled the sell, then stop. In our case the result we are seeking is,

q)allocation
2 1 4 3 2 0

The insight is to realize that the cumulative sum of the allocations reaches the sell amount and then levels off: this is a restatement of what it means to do FIFO allocation.

q)sums allocation
2 3 7 10 12 12

Next we realize that the cumulative sum of buys is the total amount available for allocation at each step.

q)sums buys
2 3 7 10 15 19f

To make this sequence level off at the sell amount, simply use &.

q)sell&sums buys
2 3 7 10 12 12f

Now that we have the cumulative allocation amounts, we need to unwind this to get the step-wise allocations. This entails subtracting successive items in the allocations list.

Wouldn't it be nice if q had a built-in function that returned the successive differences of a numeric list? There is one (deltas) and - no surprise - it involves an iterator (called Each Previous–more about that in Chapter 5).

q)deltas 1 2 3 4 5
1 1 1 1 1
q)deltas 10 15 20
10 5 5

Observe in your console display that (deltas) returns the initial item untouched. This is just what we need.

Returning to our example of FIFO allocation, we apply (deltas) to the cumulative allocation list and we're done.

q)deltas sell&sums buys
2 1 4 3 2 0f

Look ma, no loops! We shall see how to extend this to multiple sells in 6.7.9.

1.15 Dictionaries and Tables 101

After lists, the second basic data structure of q is the dictionary, which models key-value association. A dictionary is constructed from two lists of the same length using the ! operator. The left operand is the list of (presumably unique, though unenforced) keys and the right operand is the list of values. A dictionary is a first class value, just like an integer or list and can be assigned to a variable.

q)`a`b`c!10 20 30
| a| 10 |
| b| 20 |
| c| 30 |
q)d:`a`b`c!10 20 30

Observe that dictionary console display looks like the I/O table of a mathematical mapping. No coincidence.

Provided the dictionary has symbolic keys as above, we can also use dictionary definition syntax as follows. This type of dictionary is known as a record or structure in other languages.

q)([a:10; b:20; c:30])
a| 10
b| 20
c| 30

Given a key, we retrieve the associated value with the same square bracket notation as list indexing.

q)d[`a]
10

A useful class of dictionary has as keys a simple list of symbols and as values a list of lists of uniform length. We think of such a dictionary as a named collection of columns and call it a column dictionary.

q)`c1`c2!(10 20 30; 1.1 2.2 3.3) 
c1| 10  20  30  
c2| 1.1 2.2 3.3 
q)dc:([c1:10 20 30; c2:1.1 2.2 3.3])

Retrieving by key yields the associated column, which is itself a list and so can be indexed.

q)dc[`c1]
10 20 30
q)dc[`c1][0]
10
q)dc[`c2][1]
2.2

Whenever such iterated indexing of nested entities arises in q, there is an equivalent syntactic form, called indexing at depth, to make things a bit more readable.

q)dc[`c1][0]
10
q)dc[`c1; 0]
10
q)dc[`c1; 1]
20
q)dc[`c1; 2]
30

Indexing at depth notation suggests thinking of dc as a two-dimensional entity; this is reasonable in view of its display above. Let's pursue this. Whenever an index is elided in q, the result is as if every legitimate value had been specified in the omitted index position. For a column dictionary, this yields the associated column when the second slot is omitted.

q)dc[`c1;]
10 20 30
q)dc[`c2;]
1.1 2.2 3.3

Things are more interesting when the index in the first slot is elided. The result is a dictionary comprising a section of the original columns in just the specified position.

q)dc[;0] 
c1| 10 
c2| 1.1 
q)dc[;1] 
c1| 20 
c2| 2.2 
q)dc[;2] 
c1| 30 
c2| 3.3

To summarize, we have an entity that retrieves columns in the first slot and section dictionaries in the second slot. The issue is that columns are conventionally accessed in the second slot of two-dimensional things. No problem. We apply the built-in operator flip - better thought of as "transpose" - to reverse the order of indexing. We still have the same column dictionary but slot retrieval is reversed: columns are accessed in the second slot and section dictionaries are retrieved from the first slot.

q)t:flip ([c1:10 20 30; c2:1.1 2.2 3.3]) 
q)t[0; `c1] 
10 
q)t[1; `c1] 
20 
q)t[2; `c1] 
30 
q)t[0; `c2] 
1.1 
q)t[; `c1] 
10 20 30 
q)t[0;] 
c1| 10 
c2| 1.1

We emphasize that the data is still stored as a column dictionary under the covers; only the indexing slots are affected.

Observe that the console display of a flipped column dictionary is indeed the transpose of the column dictionary display and in fact looks like ... a table.

q)flip ([c1:10 20 30; c2:1.1 2.2 3.3])
c1 c2
------
10 1.1
20 2.2
30 3.3

A flipped column dictionary, called a table, is a first class entity in q.

In the table setting, the section dictionaries are called records of the table. They correspond to the rows of SQL tables. To see why, observe that the record at index 0 is effectively the horizontal slice of the table in "row" 0. Let's reexamine record retrieval, this time omitting the optional trailing semicolon from the elided second index.

q)t[0] 
c1| 10 
c2| 1.1 
q)t[1] 
c1| 20 
c2| 2.2 
q)t[2] 
c1| 30 
c2| 3.3

Looking at this syntactically, we might conclude that t is a list of record dictionaries. In fact it is, at least logically; physically a table is always stored as a collection of named columns.

Thus we have arrived at:

A q table is a flipped column dictionary.
It is also a list of record dictionaries.

While we can always construct a table as a flipped column dictionary, there is a convenient syntax that puts the names together with the columns. The notation looks a bit odd at first but it will seem more reasonable when we encounter keyed tables later.

q)([] c1:10 20 30; c2:1.1 2.2 3.3)
c1 c2 
------
10 1.1
20 2.2
30 3.3

A few notes

The square brackets are necessary to differentiate a table from a list with variable assignments as side effects.
The occurrence of : is not assignment. It is merely a syntactic marker separating the name from the column values
The column names in a table definition are not symbols, although they are converted to symbols under the covers.

1.16 qSQL 101

There are multiple ways to operate on tables. First, you can treat a table as the column dictionary that it is and perform basic dictionary operations on it. Newbies who are familiar with SQL may find it easier to use q's version of SQL-like syntax, called qSQL. In this section we explore basic qSQL features.

The fundamental qSQL operation is the select template. We say template because, unlike other q primitives, it is not evaluated right-to-left. Rather, it is syntactic sugar designed to mimic SQL SELECT. That said, we emphasize that although select does act like SQL SELECT in some respects, there is one fundamental difference. Whereas SQL SELECT operates on fields on a row-by-row basis, select performs vector operations on column lists. Insisting on thinking in rows with q tables will end in tears.

We construct a simple table for our examples.

q)t:([] c1:1000+til 6; c2:`a`b`c`a`b`a; c3:10*1+til 6)
q)t
c1   c2 c3
----------
1000 a  10
1001 b  20
1002 c  30
1003 a  40
1004 b  50
1005 a  60

The simplest form of select retrieves all the records and columns of the table by leaving unspecified which rows or columns - there is no need for the wildcard * of SQL. The select and from must occur together.

q)select from t
c1 c2 c3
----------
1000 a 10
1001 b 20
1002 c 30
1003 a 40
1004 b 50
1005 a 60

The next example shows how to specify which columns to return and optional names to associate with them.

q)select c1, val:2*c3 from t
c1 val
--------
1000 20
1001 40
1002 60
1003 80
1004 100
1005 120

We make several observations:

Result columns are separated by , and are sequenced left-to-right.
Any q expressions inside select are evaluated right-to-left, as usual.
As was the case with table definition syntax, instances of : are not assignment; rather, they are syntactic markers separating a column name to its left from the q expression to its right, which computes the column.
Arbitrary q expressions can be used to produce result columns, provided all column lengths are the same.
There are optional by and where phrases for grouping and constraints.

The next example demonstrates using the by phrase of select to perform grouping. The basic usage is similar to GROUP BY in SQL, in which the column expressions involve aggregate functions. All records having common values in the by column(s) are grouped together and then aggregation is performed within each group.

q)select count c1, sum c3 by c2 from t
c2| c1 c3 
--| ------
a | 3  110
b | 2  70 
c | 1  30

An advantage of qSQL by is that you can group on a computed column.

q)select count c2 by ovrund:c3<=40 from t
ovrund| c2
------| --
0     | 2 
1     | 4

Closely related to select is the update template. It has the same syntax as select but semantically the names to the left of : are interpreted as columns to modify (or add, if not already present). As with select, you can specify an optional where phrase, which limits the action to just those records satisfying specified constraint(s). Here is how to scale the c3 column of t just in the positions having c2 equal to `a.

q)update c3:10*c3 from t where c2=`a
c1   c2 c3 
-----------
1000 a  100
1001 b  20 
1002 c  30 
1003 a  400
1004 b  50 
1005 a  600

We stress that the operations in update are vector operations on columns, not row-by-row.

Not all of qSQL is included in the templates. For example, to sort a table ascending by column(s), use xasc with left operand the symbol column name(s) in major-to-minor order.

q)`c2 xasc t
c1   c2 c3
----------
1000 a  10
1003 a  40
1005 a  60
1001 b  20
1004 b  50
1002 c  30

1.17 Example: Trades Table

In this section we construct a toy trades table to demonstrate the power of qSQL.

A useful operator for constructing lists of test data is ?, which generates pseudo-random data. We can generate 10 numbers randomly selected, with replacement, from the first 20 integers starting at 0 (i.e., not including 20).

q)10?20         / ymmv
12 8 10 1 9 11 5 6 1 5
q)10?20
4 13 9 2 7 0 17 14 9 18
q)10?20
17 8 13 17 12 12 2 4 5 12

We can similarly generate 10 random floats between 0.0 and 100.0 (not including 100.0).

q)10?100.0
40.87545 44.9731 1.392076 71.48779 19.46509 9.059026 …

We can make 10 random selections from the items in a list.

q)10?`aapl`ibm
`aapl`aapl`aapl`aapl`ibm`aapl`ibm`ibm`ibm`ibm

Now to our trades table. Since a table is a collection of columns, we first build the columns. We apologize for using excessively short names so that things fit easily on the printed page.

First we construct a list of 1,000,000 random dates in the month of January 2025.

q)dt:2025.01.01+1000000?31

Next a list of 1,000,000 timespans.

q)tm:1000000?24:00:00.000000000

Next a list of 1,000,000 tickers chosen from AAPL, GOOG and IBM. It is customary to make these lower case symbols.

q)sym:1000000?`aapl`goog`ibm

Next a list of 1,000,000 volumes given as positive lots of 10.

q)vol:10*1+1000000?1000

As an initial cut, we construct a list of 1,000,000 prices in cents uniformly distributed within 10% of 100.0. We will adjust this later.

q)px:90.0+(1000000?2001)%100

Now collect these into a table and inspect the first 5 records. Remember, a table is a list of records so # applies.

q)5#trades:([] dt; tm; sym; vol; px)
dt tm sym vol px
------------------------------------------------
2025.01.18 0D16:36:39.679324775 goog 1650 109.72
2025.01.31 0D13:21:37.832939028 aapl 8480 100.37
2025.01.16 0D12:43:12.913133651 ibm 5970 97.96
2025.01.18 0D20:08:01.216212362 aapl 8740 108.04
2025.01.08 0D12:11:51.340559273 aapl 1710 107.41

The first thing you observe in your console display is that the trades are not in temporal order. We fix this by sorting on time within date using xasc.

q)5#trades:`dt`tm xasc trades
dt tm sym vol px
------------------------------------------------
2025.01.01 0D00:00:01.817692816 ibm 9190 101.64
2025.01.01 0D00:00:02.149555832 aapl 850 104.35
2025.01.01 0D00:00:04.553968459 ibm 9870 98.96
2025.01.01 0D00:00:06.205438077 ibm 8820 99.45
2025.01.01 0D00:00:06.734785437 aapl 1980 97.58

Now we adjust the prices. At the time of this writing AAPL was trading around 200, so we double it. We also adjust GOOG and IBM to their approximate trading ranges by scaling.

q)trades:update px:2*px from trades where sym=`aapl
q)trades:update px:1.75*px from trades where sym=`goog
q)trades:update px:2.7*px from trades where sym=`ibm
q)5#trades
dt tm sym vol px
------------------------------------------------
2025.01.01 0D00:00:01.817692816 ibm 9190 274.428
2025.01.01 0D00:00:02.149555832 aapl 850 208.7
2025.01.01 0D00:00:04.553968459 ibm 9870 267.192
2025.01.01 0D00:00:06.205438077 ibm 8820 268.515
2025.01.01 0D00:00:06.734785437 aapl 1980 195.16

This looks a bit more like real trades. Let's perform some basic queries as sanity checks. Given that both price and volume are uniformly distributed, we expect their averages to approximate the mean. Using the built-in average function (avg) we see that they do.

q)select avg px, avg vol by sym from trades
sym | px       vol     
----| -----------------
aapl| 199.9841 5009.952
goog| 175.0053 5007.141
ibm | 269.9864 4998.28

Similarly, we expect the minimum and maximum price for each symbol to be the endpoints of the uniform range.

q)select minpx:min px, maxpx:max px by sym from trades
sym | minpx    maxpx  
----| ---------------
aapl| 180      220  
goog| 157.5    192.5
ibm | 243      297

Our first non-trivial query computes the 100 millisecond bucketed volume-weighted average price VWAP. This uses the built-in binary function xbar. The left operand of xbar is an interval width and the right operand is a list of numeric values. The effect of xbar is to shove each input to the left-hand end point of the interval of specified width in which it falls. For example,

q)5 xbar til 15
0 0 0 0 0 5 5 5 5 5 10 10 10 10 10

This is useful for grouping since it effectively buckets all the values within each interval to the left end-point of that interval. Recalling that a timespan is actually an integral count of nanoseconds since midnight, to compute 100 millisecond buckets we will use xbar with an interval of 100,000,000.

We also require wavg, a binary function that computes the average of the numeric values in its right operand weighted by the values of its left operand.

q)1 2 3 wavg 50 60 70
63.33333

Now we put things together in a single query. For convenience of display, we group by bucketed time within symbol.

q)select vwap:vol wavg px by sym,bkt:100000000 xbar tm from trades
sym  bkt                 | vwap    
-------------------------| --------
aapl 0D00:00:00.000000000| 189.7   
aapl 0D00:00:00.300000000| 202.72  
aapl 0D00:00:00.600000000| 182.7   
aapl 0D00:00:00.800000000| 196.856

That's all there is to it!

Our final query involves the maximum profit (or analogously, maximum drawdown) realizable over the trading period. To understand the concept, imagine that you have a DeLorean with flux capacitor and are able to travel into the future and record historical trade results. Upon returning to the present, you are given $1,000,000 to invest with the stipulation that you can make one buy and one sell for AAPL and you are not allowed to short the stock. As a good capitalist your goal is to maximize your profit.

Restating the problem, we wish to determine the optimum time to buy and sell for the largest (positive) difference in price, where the buy precedes the sell. We state the solution as a q koan, which you should contemplate until enlightenment.

q)select max px-mins px from trades where sym=`aapl
px
--
40

Intuitively we see this answer is correct since this is the maximum difference we could achieve due to the uniform distribution.

Two hints if enlightenment is slow to dawn.

Take the perspective of looking back from a potential optimum sell
The optimum buy must happen at a cumulative local minimum; otherwise, you could back up to an earlier, lower price and make a larger profit.

1.18 File I/O 101

For this section we need to introduce the other q primitive text data type, called char. A single ASCII character is represented as that character in double quotes. Here are some examples.

q)"a"
"a"
q)" "
" "
q)"_"
"_"

The char "a" is an atom but is not the same as its symbol cousin `a.

Things go sideways with a simple list of char. Enter such a list in general form and observe the simplified display echoed on the console.

q)("s"; "t"; "r"; "i"; "n"; "g")
"string"

A simple list of char looks like a string from traditional languages and is even called a string in q. But this string is not an atom; a string isn't even a first class citizen in q. Out string is a list having count 6. And it should not be confused with its symbol cousin `string, which is an atom having count 1.

q)count "string"
6
q)count `string
1

With these preliminaries out of the way, we proceed to I/O. The way q handles I/O is Spartan. No instantiation of readers, writers, serializers and the like. We admit that the notation is funky, but you will grow to appreciate its conciseness just as a serious driver prefers a manual transmission...if you can still find one.

File I/O begins with handles. A symbolic file handle is a symbol of a particular form that represents the name of a resource on the file system. The leading : differentiates the symbol as a handle. For example,

`:path/filename

We use the following simple table in our demonstration.

q)t:([] c1:`a`b`c; c2:1.1 2.2 3.3)
q)t
c1 c2
------
a 1.1
b 2.2
c 3.3

Pick a destination to write your files. Normally in a production q/kdb allocation we would only write below the current working directory. To make things easier in this short demo, I will write to the following directory on my Mac.

/Users/jaborror/examples

You should replace this with your chosen directory when entering what follows.

To save the table t in a serialized binary data file, use the built-in function set with symbolic file handle as left operand and the source data as the right operand.

q)`:/Users/jaborror/examples/t set t
`:/Users/jaborror/examples/t

Observe that the console echoes the symbolic file handle in case of success. To read the stored data and deserialize it back into the session, use get with the symbolic file handle.

q)get `:/Users/jaborror/examples/t
c1 c2
------
a 1.1
b 2.2
c 3.3

Presto! It's out and back.

To write text data to a file we use one of the overloads of the infelicitously named 0: operator. The key idea is that q considers a text file to correspond to a list of strings, one string per file record. We supply 0: with a symbolic file handle as its left operand and a list of strings (i.e., a list of lists of char) in the right operand.

q)`:/Users/jaborror/examples/life.txt 0: ("Meaning";"of";"life")
`:/Users/jaborror/examples/life.txt

To read a text file as a list of strings, use read0 with the symbolic handle.

q)read0 `:/Users/jaborror/examples/life.txt
"Meaning"
"of"
"life"

And now, what everyone is waiting for: writing and reading csv files. Hold on to your hats, as this uses three different overloads of 0:. One to prepare the tables as text; the one we already met to write text files; and one to read formatted text files. Certainly a regrettable naming convention.

Preparing a table as csv text is simple; q handles the quoting and escaping of special characters. Apply 0: with the defined constant csv as left operand and the table in the right operand.

q)csv 0: t
"c1,c2"
"a,1.1"
"b,2.2"
"c,3.3"

Your console display shows the table properly prepared as strings. Now compose this result with the previous overload of 0: and write it out. As a check, we use read0 to read back the text file as a list of strings.

q)`:/Users/jaborror/examples/t.csv 0: csv 0: t
`:/Users/jaborror/examples/t.csv
q)read0 `:/Users/jaborror/examples/t.csv
"c1,c2"
"a,1.1"
"b,2.2"
"c,3.3"

Finally, we demonstrate the third overload of 0: to parse the formatted csv file into the q session as a table. The right operand is a symbolic file handle. The left operand is a control list with two items. The first item is a string of upper case characters indicating the types of each field within the text row.

The second item of the control list is the field separation character - in our case this is ,. This separator char should be enlisted if there are column headers in the first row of the file, as in our case. These headers are used as table column names. For our example we have,

q)("SF"; enlist ",") 0: `:/Users/jaborror/examples/t.csv
c1 c2
------
a 1.1
b 2.2
c 3.3

Here S and F indicate that there are two fields, having types symbol and float. The separator is an enlisted ,.

Yes, the naming and notation is obscure. But you have to admit that file I/O can't get much simpler.

1.19 Interprocess Communication 101

For this section, you will need two open q sessions, best done on the same machine. We recommend that this machine be one that is not encumbered with enterprise security. Choose one session to be your "server" and open a port with the command \p (note lower case) followed by the port number. To verify that the port is open, execute the naked command \p and check that it echoes a 32-bit int of the port you opened.

q)\p 5042 / on server
q)\p
5042i

The syntax of Interprocess Communication (IPC) is similar to that of File I/O. A symbolic network handle is a symbol of a particular form that identifies the name of a resource on the network. For our purposes, it suffices to consider a network handle of the simplest form.

`:localhost:5042

The leading : in the symbol identifies it as a symbolic handle. To the left of the second : is the name of the network resource - in this case, the machine on which the q session is running. To the right of : is a (presumably open) port on the destination machine that will be used for TCP/IP communication.

To open a connection, use a symbolic handle as argument to hopen and store the result in a variable, traditionally called h. Do that now in your "client" session after ensuring that the specified port is open in the "server" session.

q)h:hopen `:localhost:5042      / on client

The variable h is called an open handle. It represents a function for sending a request to the server and receiving the result of that request. Now we're ready to party.

There are three ways to send requests from the client to the server, only one of which is safe for production applications. For demonstration purpose (only), we show the simplest, which should only be used in development environments. When invoked with a string - i.e., a list of char - argument, the handle function h sends that string to the server, where it is executed, and any result is synchronously returned from the application of h.

q)h "6*7"       / on client
42

Clearly this isn't safe, as arbitrary text could be sent for nefarious purposes.

We can also send a function to the remote server for application.

q)h ({x*x}; 3)
9

This is also unsafe since am arbitrary function can be passed.

A safer way to make requests to the server is to invoke h with a list containing the name of a function that (presumably) exists on the server, followed by arguments to that function. When h is invoked with such a list argument, it (synchronously) causes the server to apply the named function to the transmitted arguments, and then returns any result from the server is its own output. This corresponds to call by name in a traditional remote procedure call. It is safer since the server can inspect the symbolic function name and determine whether the requesting user is authorized to execute it

On your server process, create a simple function of two arguments.

q)f:{x*y}   / on server

On your client process, invoke h with a list containing the symbolic name of the remote function followed by its two arguments.

q)h (`f; 6; 7)   / on client
42

Observe that nothing is displayed on the server console since the function application there returns its result to the client. To close the connection with the server, flush buffers and free resources, apply hclose to the open handle.

q)hclose h   / on client

IPC doesn't get any easier.

1.20 Example: Asynchronous Callbacks

The IPC mechanism of q does not have callbacks built in but it is powerful enough that we can create callbacks ourselves. We assume that you have started separate client and server q sessions and have opened the connection from the client to the server, as in the previous section.

Heretofore, calls to the server were synchronous, meaning that at the point of the remote call, the client blocks until the requested work on the server completes and the result is returned. It is also possible to make the remote call asynchronous. In this case, the client does not block: the application of the open handle returns immediately.

In order to demonstrate this, we have to come clean about what is really in the open handle h. You can see for yourself by displaying h from an open connection.

q)h:hopen `:localhost:5042
q)h
6i

Your result will probably not match this but it will be a (non-long) integer. Yes, an open handle is just a positive 32-bit integer. When this (positive) integer is applied as a function, the call is synchronous. To make an asynchronous call, negate the value in h - i.e., neg h - and use this with function application syntax. Seriously.

Since nothing will be displayed in the client session, it helps to display progress on the server as the request is performed. Create the function echo in the server session.

q)echo:{show x}   / on server

Now make an asynchronous remote call to echo from the client.

q)(neg h) (`echo; 42)   / on client
q)

Observe on your q consoles that the client application returns immediately with no result and that the server displays the progress message.

Now to callbacks. We begin by instrumenting a function rsvp on the server that, when invoked remotely, will call back to the client. It will receive two parameters: it's own argument and the symbolic name of the client function to call.

q)rsvp:{[arg;cb] ..}     / on server

We initially invoke the server's (show) with the passed arg to indicate that we are hard at work on the transmitted data.

q)rsvp:{[arg;cb] show arg;}

Now for the big moment. To make the return call, we need the server's open handle of the connection for the remote call it is processing. This is conveniently placed in the q system variable .z.w ("who" called) for the duration of each remote call. We use it to make an asynchronous remote call (hence the neg) over the caller's handle, passing the provided callback name and our arduously computed result r.

q)rsvp:{[arg;cb] show arg; r:1+arg; (neg .z.w) (cb; r);}

In the final step, we display another progress message on the server console indicating the remote call has completed. Since this function returns its actual result remotely, we end its body with ; to ensure that it returns nothing locally.

q)rsvp:{[arg;cb] show arg; r:1+arg; (neg .z.w) (cb; r); show `done;}

We turn to the client side and create echo to serve as the function called back for this demonstration.

q)echo:{show x} / on client

All that remains is to fire off the remote asynchronous call from the client. We pass to the client open handle a list containing: the name of the remote function; the argument for the remote function; and the name of our own function to be called back during the remote computation. Be sure to do it asynchronously else you will get deadlocks.

q)(neg h) (`rsvp; 42; `echo)    / on client

Provided all went well, the server console will display:

q)42
`done

The client console should display:

q)(neg h) (`rsvp; 42; `echo)
q)43

And there we have it. Callbacks built from scratch in q using a few lines of code.