Cookbook/InterfacingWithC

From Kx Wiki
Jump to: navigation, search

The wiki is moving to a new format and this page is no longer maintained. You can find the new page at code.kx.com/q//interfaces/c-client-for-q/.

The wiki will remain in place until the migration is complete. If you prefer the wiki to the new format, please tell the Librarian why.

Contents

Interfacing with Kdb+ from C

Introduction

The best way to understand the underpinnings of KDB+, and to interact with it from C is to start with the header file available from here.

This is the file you will need to include in your C or C++ code in order to interact with Kdb+ from a low-level.

Note that the k struct changed with the release of v3.0, and if you are compiling using the c library (c.o/c.dll) stamped on or after 2012.06.25 you should ensure you use the correct k struct by defining KXVER accordingly. e.g. gcc -D KXVER=3 ... If you need to link against earlier releases of the c library, you can obtain those files from the svn repository [1], revision 1442.

Lets explore the basic types and their synonyms that you'll commonly encounter when programming at this level. First though, it is worth noting the size of data types in 32 versus 64 bit operating systems to avoid common mistake.

64 bit Linux is an "LP64" operating system. This means that an int is 32 bits, while long, long long and pointers are 64 bits. The same is true for Solaris, AIX, HP-UX, and Mac OS X. Microsoft Windows on the EM64T and AMD64 architectures use a different model, LLP64. This is an attempt to maintain better compatibility with existing 32 bit source code which assumes long will be exactly 32 bits. In the LLP64 model, int and long are 32 bit while long long and pointers are 64 bit. Single precision floating point and double precision are always 32 and 64 bits respectively in all the data type models (ignoring extended or non-standard floating point formats on certain architectures).

To provide succinct composable names, the Kdb+ header defines synonyms for the common types as in the following table:

Type Synonym
16 bit int H
32 bit int I
64 bit int J
char* S
unsigned char G
char C
32 bit float E
64 bit double F
void V

With this basic knowledge we can now tackle the types available in Kdb+ and their matching C types and accessor functions provided in the C interface. We will see shortly how the accessor functions are used in practice.

Kdb+ type name Kdb+ type number Encoded type name C type Size in bytes Interface list accessor function
mixed list 0 - K - kK
boolean 1 KB char 1 kG
guid 2 UU U 16 kU
byte 4 KG char 1 kG
short 5 KH short 2 kH
int 6 KI int 4 kI
long 7 KJ int64_t 8 kJ
real 8 KE float 4 kE
float 9 KF double 8 kF
char 10 KC char 1 kC
symbol 11 KS char* 4 or 8 kS
timestamp 12 KP int64_t 8 kJ
month 13 KM int 4 kI
date 14 KD int 4 kI (days from 2000.01.01)
datetime 15 KZ double 8 kF (days from 2000.01.01)
timespan 16 KN int64_t 8 kJ (nanoseconds)
minute 17 KU int 4 kI
second 18 KV int 4 kI
time 19 KT int 4 kI (milliseconds)
table/flip 98 XT - - x->k
dict/table with primary keys 99 XD - - kK(x)[0] for keys and kK(x)[1] for values
error -128 - char* 4 or 8 x->s

Note that the type numbers given are for vectors of that type. For example, 9 for vectors of the Kdb+ type float. By convention, the negative value is a scalar: -9 is the type of a scalar float value.

The K object structure

The Kdb+ types are all encapsulated at the C level as "K objects". Recall that k is the low-level language underlying the q language in Kdb+. K objects are all instances of the following structure (note this is technically defining K objects as pointers to the the k0 structure but we'll conflate the terms and refer to K objects as the actual instance).

// For v2.8 and prior, it is defined as
typedef struct k0 {
  I r;                   // The object's reference count
  H t, u;                // The object's type and attribute flags
  union {                // The data payload is contained within this union.
     // The scalars are held in the following members:
     G g;
     H h;
     I i;
     J j;
     E e;
     F f;
     S s;
     // The following members are used for more complex data.
     struct k0*k;
     struct {
       I n;
       G G0[1];
     };
  };
}*K;

//For v3.0 and later, it is defined as
typedef struct k0{
  signed char m,a,t; // m,a are for internal use.
  C u;
  I r;
  union{G g;H h;I i;J j;E e;F f;S s;struct k0*k;struct{J n;G G0[1];};};
}*K;

As an exercise, it is instructive to count the minimum and maximum number of bytes a K object can use on your system taking into account any padding or alignment constraints.

So, given a K object x, we can use the accessors noted in the table above to access elements of the object. For example, given a K object containing a vector of floats, we can can access kF(x)[42] to get the 42nd element of the vector. For accessing scalars, use the following accessors:

byte x->g
short x->h
int x->i
long x->j
real x->e
float x->f
symbol x->s

Examining K Objects

If you are in a situation where you do know beforehand the type of the K objects, or you use to write a function that works over numerous types, it is useful to dispatch based on the type flag x->t for some K object x.

If x->t is negative then the object is a scalar and we should use the scalar accessors noted above. If x->t is greater than zero then we use the vector accessors as all the elements are of the same type (eg. x->t == KF for a vector of Kdb+ floats).

A more interesting case is where x->t is exactly zero. This means that the K object contains a mixed list of other K objects. Each element in the list is a pointer to another K object. To access each element of the object x we use the kK object accessor. For example: kK(x)[42] to access the 42nd element of the mixed list.

Nulls and Infinities

The next table provides the null and infinite immediate values for the Kdb+ types. These are constants defined in k.h.

Type Null Infinity
short 0xFFFF8000 (nh) 0x7FFF (wh)
int 0x80000000 (ni) 0x7FFFFFFF (wi)
long 0x8000000000000000 (nj) 0x7FFFFFFFFFFFFFFF (wj)
float log(-1.0) on Windows or (0/0.0) on Linux (nf) -log(0.0) in Windows or (1/0.0) on Linux (wf)

Null objects can be created using ks(""),kh(nh),ki(ni),kj(nj),kp(" "), etc. A null guid can be created with U g={0};ku(g);

Connecting to a Kdb+ server

We use the int khpu(host, port,username); function to connect to a Kdb+ server. Note you must call khpu before generating any K data, and the very first call to khpu must not be concurrent to other khpu calls. To initialise memory without making a connection, use khp("",-1);

It is highly recommended to use khpu and supply a meaningful username as this will help server administrators identify a user's connection.

The khp,khpu and khpun functions are for use in stand-alone applications only; they are not for use within a kdb+ server via a shared library. Hence, to avoid potential confusion, these functions have been removed from more recent releases of kdb+.

A timeout can be specified with function khpun, e.g.

 int c=khpun("localhost",1234,"myname:mypassword",1000); // timeout in mS

Return values for khp/khpu/khpun are:

 >0 - active handle
  0 - authentication error
 -1 - error
 -2 - timeout(khpun case)

Note that with the release of c.o with kdb+2.6, c.o now tracks whether the connection type (pre2.6, or 2.6+), and hence to close the connection you must call kclose (instead of close/closeSocket) - this will clean up the connection tracking and close the socket.

The k function is used to send messages over the connection. If a positive handle is used then the call is synchronous otherwise it is an asynchronous call.

For example:

int c = khpu("localhost", 1234,"myusername:mypassword"); // Connect to a Kdb+ server on the localhost port 1234.
if(c<=0) {perror("Connection error");return;}
K r = k(-c,"a:2+2",(K)0);             // Asynchronously set a to be 4 on the server.
r = k(c,"b:til 1000000",(K)0);        // Synchronously set b to be a list up to 1000000.
r = k(c,(S)0);                        // Read incoming data(blocking call)

Note that the object returned from an async set call must not be passed to r0.

There is no timeout argument for the k(handle,...,(K)0) call, but you can use socket timeouts as described below, or if you are reading incoming async msgs only, you can use select to determine whether the handle has data pending as follows

#include"k.h"
#include<stdio.h>
#include<stdlib.h>
#include<string.h>
#include<unistd.h>
#include<sys/types.h>
#include<sys/socket.h>
#include<netinet/in.h>
#include<arpa/inet.h>
int main(){
    int retval;
    K x,r;
    int fd=khp("localhost",9999); // In a production system, check the return value
    fd_set fds;
    struct timeval tv;
    while(1){
        tv.tv_sec=5;
        tv.tv_usec=0;
        FD_ZERO(&fds);
        FD_SET(fd,&fds);
        retval=select(fd+1,&fds,NULL,NULL,&tv);
        if(retval==-1)
             perror("select()"),exit(1);
        else if(retval){
            printf("Data is available now.\n");
            if(FD_ISSET(fd,&fds)){
                x=k(fd,(S)0);
                if(x){
                  if(-128==x->t){printf("Error occurred:%s\n",x->s);return;}
                  printf("%d\n",x->i);
                  r0(x);
                }
                //else connection closed
            }
        }
        else
            printf("No data within five seconds.\n");
    }
    kclose(fd);
    return 0;
}

Socket Timeouts

There are a number of reasons not to specify/implement timeouts - typically these will be hit at the least convenient of times when under load from e.g. sudden increase in trading volumes, and cascading timeouts can rapidly bring systems down and/or waste server resources. However, if you are convinced they are the only solution for your problem scenario, the following code may help you. Note that in the event of a timeout, you must close the connection.

#if defined(_WIN32) || defined(__WIN32__)
V sst(I d,I sendTimeout,I recvTimeout){
  setsockopt(d,SOL_SOCKET,SO_SNDTIMEO,(char*)&sendTimeout,sizeof(I));
  setsockopt(d,SOL_SOCKET,SO_RCVTIMEO,(char*)&recvTimeout,sizeof(I));}
#else
V sst(I d,I sendTimeout,I recvTimeout){
  struct timeval tv;tv.tv_sec=sendTimeout/1000;tv.tv_usec=sendTimeout%1000000;
  setsockopt(d,SOL_SOCKET,SO_SNDTIMEO,(char*)&tv,sizeof(tv));
  tv.tv_sec=recvTimeout/1000;tv.tv_usec=recvTimeout%1000000;
  setsockopt(d,SOL_SOCKET,SO_RCVTIMEO,(char*)&tv,sizeof(tv));}
#endif

// usage
int c=khpun("localhost",1234,"myname:mypassword",1000); // connect timeout 1000 mS
if(c>0) sst(c,30000,45000); // timeout sends with 30s, receives with 45s

Managing memory and Reference Counting

Although in Kdb+ memory is automatically managed for the programmer, when interfacing from C or C++ we must (as is traditional in those languages) manually manage memory. The following functions are provided to interface with the Kdb+ memory manager:

Increment the object's reference count r1(x)
Decrement the object's reference count r0(x)
Free up memory allocated for the thread's pool m9()
Set whether interning symbols uses a lock setm(f)

A reference count indicates the usage of an object, allowing the same object to be used by more than one piece of code.

If you create a k object through one of the 'generator' functions (ki,kj,knk,etc), you automatically have a reference to that object. Once you have finished using that object, you should call r0(x).

e.g.

r0(ki(5));

creates and immediately destroys an integer object.

In the case of a function being called from Q

K myfunc(K x)
{
    return ki(5);
}

the object is returned to Q, and Q will eventually decrement the reference count.

In this scenario, a function being called from Q, the arg x from Q is passed to the c function. If it is to be returned to Q, the reference count must be incremented.

K myfunc(K x)
{
    return r1(x);
}

The function k, as in

K r=k(handle,"functionname",params,(K)0);

requires a little more explanation.

If the handle is >=0, it is a generator function, and can return 0 (indicating a network error) or a pointer to a k object. If that object has type -128, it indicates an error, accessible as a null terminated string in r->s. When finished using this object, it should be freed through calling r0(r).

If the handle is <0, this is for async messaging, and the return value can be either 0 (network error) or non-zero (ok). This result should NOT be passed to r0(r).

K objects as parameters to the k function call will have their reference counts automatically decremented during the return from that call. To continue to use the object later in that c function, after the k call, increment the reference count before call, e.g.

K r=k(handle,"functionname",r1(param),(K)0);

It is absolutely vital to increment and decrement when adding or removing references to values that should be managed by the Kdb+ runtime in order to avoid memory leaks or access faults due to double frees.

Note that k objects must be freed from the thread they are allocated within, and m9() should be called when the thread is about to complete, freeing up memory allocated for that thread's pool. Furthermore, to allow symbols to be created in other threads, setm(1) should be called from the main thread before any other threads are started.

When a k object is created, it usually has a reference count of 0 - exceptions are common constants such as (::) which may vary in their current reference count, as they may be used by other areas of the c-api library or kdb+. If r0 happens to be passed a k object with a reference count of 0, that object's memory is freed (returned to an internal pool). Be aware that if a reference count is >0, you should very likely NOT change the data stored in that object as it is being referenced by another piece of code which may not expect the object to change - in this case, create a new copy of the object, and change that.

If in doubt, the current reference count can be seen in c with

printf("Reference count for x is %d\n",x->r);

and in q with

    -16!x

Creating scalar values

To create scalar values the following functions are available. Function ka creates an atom of the given type, and the rest create an atom with the given value:

Create an atom of type K ka(I);
Create a boolean K kb(I);
Create a guid K ku(U);
Create a byte K kg(I);
Create a short K kh(I);
Create an int K ki(I);
Create a long K kj(J);
Create a real K ke(F);
Create a float K kf(F);
Create a char K kc(I);
Create a symbol K ks(S);
Create a timestamp K ktj(-KP,J);
Create a time K kt(I);
Create a date K kd(I);
Create a timespan K ktj(-KN,J);
Create a datetime K kz(F);

An example of creating an atom:

K z = ka(-KI);
z->i = 42;

Equivalently:

K z = ki(42);

Creating lists

To create lists use the following functions:

Create a simple list K ktn(type,length);
Create a mixed list K knk(n,x,y,z);

Length must be in the range 0..2 billion. It may NOT be 0N (ni, null int).

For example to create an integer list of 5 we say ktn(KI,5). A mixed list of 5 elements can be created with ktn(0,5) but note that each element MUST be initialized before further usage; A convenient shortcut to creating a mixed list when all elements already exist upon creation of the list is to use knk, e.g. knk(2,kf(2.3),ktn(KI,10)). As we've noted, the type of a mixed list is 0, and the elements are pointers to other K objects - hence it is mandatory to initialize those n elements either via knk params, or explicitly setting each element when created with ktn(0,n) .

To append to ("join") lists we use the following:

Join an atom to a list K ja(K*,V*);
Join a string to a list K js(K*,S);
Join another K object to a list K jk(K*,K);
Join another K list to the first K jv(K*,K);

the join functions assume there are no other references to the list, as the list may need to be reallocated during the call. In case of reallocation passed K* pointer will be updated to refer to new K object and returned from the function.

K x=ki(42);
K list=ktn(0,0);
jk(&list,x); // append a k object to a list

K vector=ktn(KI,0);
int i=2;
ja(&vector,&i); // append a primitive int to an int vector

K syms=ktn(KS,0);
S sym=ss("IBM");
js(&syms,sym); // append an interned symbol to a symbol vector

K more=ktn(KS,2);
kS(more)[0]=ss("INTC");
kS(more)[1]=ss("GOOG");
jv(&syms,more); // append a vector with two symbols to syms

Strings and datetimes

Strings and datetimes are special cases and extra utility functions are provided:

Create a string K kp(string);
Create a string of length n K kpn(string, n);
Intern a string S ss(string);
Intern n chars from a string S sn(string, n);
Create a Kdb+ date from an integer int dj(int n);
Encode a year/month/day as an int I ymd(year,month,day);

Recall that unix time is the number of seconds since 1970.01.01T00:00:00 while kdb+ time types have an epoch of 2000.01.01T00:00:00 - as can be seen with the following q code

q)`long$`timestamp$2000.01.01
0j
q)`int$2000.01.01
0

Utilities to convert between unix and Kdb+ time types may be defined as below.

F zu(I u){return u/8.64e4-10957;}   // kdb+ datetime from unix
I uz(F f){return 86400*(f+10957);}  // unix from kdb+ datetime
J pu(I u){return 8.64e13*(u/8.64e4-10957);} // kdb+ timestamp from unix, use ktj(Kj,n) to create timestamp from n
I up(J f){return (f/8.64e13+10957)*8.64e4;}  // unix from kdb+ timestamp
struct tm* lt(int kd) { time_t t = uz(kd); return localtime(&t); }
struct tm* lt_r(int kd, struct tm* res) { time_t t = uz(kd); return localtime_r(&t, res); } 
struct tm* gt(int kd) { time_t t = uz(kd); return gmtime(&t); }
struct tm* gt_r(int kd, struct tm* res) { time_t t = uz(kd); return gmtime_r(&t, res); }
char* fdt(struct tm* ptm, char* d) { strftime(d, 10, "%Y.%m.%d", ptm); return d; }
void tsms(unsigned ts,char*h,char*m,char*s,short*mmm) {*h=ts/3600000;ts-=3600000*(*h);*m=ts/60000;ts-=60000*(*m);*s=ts/1000;ts-=1000*(*s);*mmm=ts;}
char* ftsms(unsigned ts, char* d){char h, m, s; short mmm; tsms(ts, &h, &m, &s, &mmm); sprintf(d, "%02d:%02d:%02d.%03d", h, m, s, mmm); return d;}

What's the difference between a symbol and a char vector?

A symbol is a pointer to a location in an internal map of strings; that is symbols are interned zero terminated strings. On the other hand, a char vector is similar to an int vector and is instead a counted K vector as usual.

In order to create a symbol we must intern a string and then use the resulting pointer in further expressions.

K someSymbol = ks(ss("some symbol"));
K nullSymbol = ks("");

(As of version 2.4, ks will call ss internally. It is still necessary to intern strings before storing them in a symbol vector, e.g. kS(v)[i] = ss("some symbol");.)

Creating dictionaries and tables

We can create dictionaries and tables from C using the following functions:

Create a dict K xD(K,K);
Create a table from a dict K xT(K);
Create a simple table from a keyed table K ktd(K);

Recall that a dictionary is a K object of type 99. It contains a list of two K objects; the keys and the values. We can use kK(x)[0] and kK(x)[1] to get these contained data. Also recall that a simple table (a "flip") is a K object of type 98. In terms of the K object, this is an atom that points to a dictionary. This means that to access the columns we can use the kK(x->k)[0] accessor and the kK(x->k)[1] for the values.

A table with primary keys is a dictionary which maps a simple table to another simple table. The following example shows the steps to create a table with primary keys:

K maketable(){
  K c,d,e,v,key,val;
/* table of primary keys */
  c=ktn(KS,1);kS(c)[0]=ss("sid");
  d=ktn(KS,3);kS(d)[0]=ss("ibm");kS(d)[1]=ss("gte");kS(d)[2]=ss("kvm");
  v=knk(1,d);
  key=xT(xD(c,v));
/* table of values */
  c=ktn(KS,2);kS(c)[0]=ss("amt");kS(c)[1]=ss("date");
  d=ktn(KI,3);kI(d)[0]=100;kI(d)[1]=300;kI(d)[2]=200;
  e=ktn(KD,3);kI(e)[0]=2;kI(e)[1]=3;kI(e)[2]=5;
  v=knk(2,d,e);
  val=xT(xD(c,v));
  return xD(key,val);
}

Although we can thus access the data using the already introduced accessors, you many find it easier to first convert it to a simple table before manipulating it in C.

/*
   Get a keyed table by executing a query.
*/
K x = k(h, "select sum size by sym", (K)0);
if(!x) perror("network error");
if(-128==x->t) { printf("Error occurred:%s\n",x->s);return;}
/*
   Convert the result to a simple table.
   NB. x is no longer valid and has been deallocated.
*/
K y=ktd(x);

/*
   Note that if the ktd conversion fails for any reason,
   it returns 0 and x is not freed.
   since 2011-01-27, ktd always decrements ref count of input.
*/
if (0 == y)
    (void) printf("x is still a keyed table because the conversion failed.");
else
    (void) printf("y is a simple table and x has been deallocated.");

Bulk transfers

A kdb+tick feed handler can send one record at a time, like this

I kdbSocketHandle = khpu("localhost", 5010, "username");
if (kdbSocketHandle > 0)
{
    K row = knk(3, ks((S)"ibm"), kf (93.5), ki(300));
    K r = k(-kdbSocketHandle, ".u.upd", ks((S)"trade"), row, (K)0);
    if(!r) { perror("network error"); return;}
    kclose(kdbSocketHandle);
}

or send many record at a time:

int n = 100;
S sid[] = {"ibm","gte","kvm"};
K x = knk(3, ktn(KS, n), ktn(KF, n), ktn(KI, n));
for(int i=0; i<n ; i++) {
    kS(xK[0])[i] = ss(sid[i%3]);
    kF(xK[1])[i] = 0.1*i;
    kI(xK[2])[i] = i;
}
K r = k(-kdbSocketHandle, ".u.upd", ks((S)"trade"), x, (K)0);
if(!r) perror("network");

This example assumes rows with three fields, symbol, price and size.

Error Signaling and Catching

To signal an error from your C code use the function krr(S);. A convenience function orr(S) can be used to signal system errors. It is similar to krr(S), but it appends a system error message to the user provided string before passing it to krr. Note that krr does not create a copy of the string passed to it. It is the user's responsibility to ensure the string is valid for the expected lifetime of the error.


To catch an error code from the results of a call to r=k(h, ..), check the return type. If it is 0 (null), then a network error has occurred. If it has type -128, then the r->s will point to the error string.

K r=k(handle, "f", arg1, arg2, NULL);
if(-128==r->t)
  printf("error string: %s\n", r->s);

under some network error scenarios, errno can be used to obtain the details of the error.

e.g. perror("network");

Return Values

If your own c function being called from q has nothing to return to q, it can return (K)0, e.g.

K doSomething(K x)
{
    // do something with x;
    return (K)0;
}

From a standalone c app, it can sometimes be convenient to return the idendity function (::). This atom can be created with

K identity(){
  K id=ka(101);
  id->g=0;
  return id;
}

Callbacks

The void sd0(I); and K sd1(I, K(*)(I)); functions are for use with callbacks and are available only within kdb+ itself, i.e. used from a shared library loaded into kdb+. The value of the file descriptor passed to sd1 must be 0 < fd < 1024, and 1021 happens to be the maximum number of supported connections (recalling 0,1,2 used for stdin,stdout,stderr).

sd1(d,f);

Puts the function K f(I d){..} on the q main event loop given a socket d (or -d for nonblocking). The function f should return (K)0 or a pointer to a kobject and its reference count will be decremented.

sd0(d);

Removes the callback on that socket.

On Linux, eventfd can be used with sd1 and sd0. e.g. given a file efd.c

// compile with
// gcc -shared -m64 -DKXVER=3 efd.c -o efd.so -fPIC
// or
// g++ -shared -m64 -DKXVER=3 efd.cpp -o efd.so -fPIC
#include"k.h"
#include<stdio.h>
#include<sys/eventfd.h>
#include<unistd.h>
#ifdef __cplusplus
extern"C"{
#endif
K callback(I d){K r;J a;R -1!=read(d,&a,8)?r=k(0,(S)"onCallback",ki(d),kj(a),(K)0),r->t==-128?krr(r->s),r0(r),(K)0:r:(sd0(d),orr((S)"read"));}
K newFd(K x){I d;R x->t!=-KJ?krr((S)"type"):(d=eventfd(x->j,0))==-1?orr((S)"eventfd"):sd1(d,callback);}
K writeFd(K x,K y){R x->t!=-KI||y->t!=-KJ?krr((S)"type"):-1!=write(x->i,&y->j,8)?0:(sd0(x->i),orr((S)"write"));}
#ifdef __cplusplus
}
#endif

and combined with appropriate q code, e.g.

q)newFd:(`$"./efd")2:(`newFd;1)
q)writeFd:(`$"./efd")2:(`writeFd;2)
q)fd:newFd 0 / arg is start value of eventfd counter
q)onCallback:{0N!(x;y)}
q)writeFd[fd;3] / increments the eventfd counter by 3, triggering the callback later

this demonstrates the deferred invocation of onCallback until kdb+ has at least finished processing the current handle/script. In situations where you can't hook a feedhandler's callbacks directly into sd1, on Linux eventfd may be a viable option for you. Callbacks from sd1 are executed on the main thread of kdb+.

New in kdb+ 3.0 2013.04.04: K sd0x(I d,I f) same functionality as sd0(I d) but f specifies whether to close d. sd0 closes d.

Windows developers may be interested in https://github.com/ncm/selectable-socketpair

Serialization/Deserialization

The K b9(I,K) and K d9(K) functions serialize and deserialize K objects.

b9(preserveEnumerations,kObject);

will generate a K byte vector which contains the serialized data for kObject. Since kdb+ v3.0, for shared libraries loaded into kdb+ the value for preserveEnumerations must be -1. For standalone applications binding with c.o/c.dll or shared libraries prior to v3.0 the values for preserveEnumerations can be

0 - unenumerate, block serialization of timespan and timestamp (For working with kdb+ versions prior to 2.6).
1 - retain enumerations, allow serialization of timespan and timestamp. (Useful for passing data between threads).
2 - unenumerate, allow serialization of timespan and timestamp
3 - unenumerate, compress, allow serialization of timespan and timestamp
d9(kObject);

will deserialize the byte stream in kObject returning a new kObject. Byte stream passed to d9 is not altered in any way. If you are concerned that the byte vector that you wish to deserialize may be corrupted, call okx to verify it is well formed first.

  unsigned char bytes[]={0x01,0x00,0x00,0x00,0x0f,0x00,0x00,0x00,0xf5,0x68,0x65,0x6c,0x6c,0x6f,0x00}; // -8!`hello
  K r,x=ktn(KG,sizeof(bytes));
  memcpy(kG(x),bytes,sizeof(bytes));
  int ok=okx(byteVector);
  if(ok){
    r=d9(byteVector);
    r0(x);
  }
  else
    perror("bad data");

Note that b9(I,K) cannot serialize objects where x->t == -128. Although a byte vector is returned, passing it to d9(K) results in a return of NULL. Therefore, test the object returned from k(...) immediately for an error type before any logic might pass it along.

Miscellaneous

The K dot (K x, K y) function is the same as the q function .[x;y].

q).[{x+y};(1 2;3 4)]
4 6

The dynamic link, K dl(V* f, I n), function takes a C function that would take n K objects as arguments and return a new K object and returns a q function. It is useful, for example, to expose more than one function from an extension module:

#include "k.h"
Z K1(f1){R r1(x);}
Z K2(f2){R r1(y);}
K1(lib){K y=ktn(0,2);x=ktn(KS,2);xS[0]=ss("f1");xS[1]=ss("f2");
  kK(y)[0]=dl(f1,1);kK(y)[1]=dl(f2,2);R xD(x,y);}

Alternately, for simpler editing of your `lib API:

#define sdl(f,n) (js(&x,ss(#f)),jk(&y,dl(f,n)))
K1(lib){K y=ktn(0,0);x=ktn(KS,0);sdl(f1,1);sdl(f2,2);R xD(x,y);}


With the above compiled into lib.so:

q).lib:(`:lib 2:(`lib;1))`
q).lib.f1 42
42
q).lib.f2 . 42 43
43

Debugging with gdb

It can be a struggle printing K values from a debugger, but you can call the handy k.h macros in gdb like xt, xC, xK..

If your client is an extension, you might get away with p k(0,"show",r1(x),(K)0)


This section of the gdb manual helped me.

 Now, we compile the program using the gnu C compiler, gcc. We pass the -gdwarf-21 and -g3 flags to ensure the compiler includes information about   preprocessor macros in the debugging information.

     $ gcc -gdwarf-2 -g3 sample.c -o sample
     $
 Now, we start gdb on our sample program:

     $ gdb -nw sample
     GNU gdb 2002-05-06-cvs
     Copyright 2002 Free Software Foundation, Inc.
     GDB is free software, ...
     (gdb)

I found that all i needed was

gcc -g3 client.c -o client
gdb ./client

.. get signal, go up stack frame with up:

Thread 1 "sdl" received signal SIGSEGV, Segmentation fault.
0x000000000040711b in nx ()
(gdb) up
#1  0x0000000000407411 in nx ()
(gdb)  
#2  0x0000000000407411 in nx ()
 (gdb) 
#3  0x0000000000408a15 in b9 ()
(gdb) 
#4  0x0000000000409ac2 in ww ()
(gdb) 
#5  0x0000000000409d33 in k ()
(gdb) 
#6  0x000000000040410d in main (n=1, v=0x7fffffffdf68) at sdl.c:108
108     }else if(e.type==SDL_USEREVENT){K x=e.user.data1;A(!xt);A(xn==2);k(-c,"{value[x]y}",xK[0]->s,xK[1],(K)0);}

Now use k.h macros!

(gdb) p xt
$20 = 0 '\000'
(gdb) p xn
$21 = 2

so it's a list of K. show two elements:

(gdb) p xK[0]->t
$23 = -11 '\365'
(gdb) p xK[0]->s
$24 = (S) 0x2078b98 "blink"
(gdb) p xK[1]->t
$25 = -7 '\371'
(gdb) p xK[1]->j
$27 = 0

which is a bit easier than:

(gdb) p *(((K*)(x->G0))[0])
$14 = {m = 0 '\000', a = 1 '\001', t = -11 '\365', u = 0 '\000', r = 0, {g = 152 '\230', h = -29800, i = 34048920, j = 34048920, 
   e = 9.95829503e-38, f = 1.6822401649996936e-316, s = 0x2078b98 "blink", k = 0x2078b98, {n = 34048920, G0 = ""}}}
(gdb) p *(((K*)(x->G0))[1])
$13 = {m = 0 '\000', a = 0 '\000', t = -7 '\371', u = 0 '\000', r = 0, {g = 0 '\000', h = 0, i = 0, j = 0, e = 0, f = 0, s = 0x0, k = 0x0, 
   {n = 0, G0 = "\002"}}}

Windows and the loadlibrary api

The kdb+ multithreaded c library (c.dll) uses static thread local storage (TLS), and is incompatible with the loadlibrary win32 api. If you are writing an excel plug-in, this point is relevant to you, as loading of the plug-in uses this mechanism. See

http://support.microsoft.com/kb/118816

for details. When trying to use the library, the problem manifests itself as a crash during the khpu() call.

Hence Kx also provides a single threaded version of this library as cst.dll, which does NOT use TLS. To use this library, download cst.dll and cst.lib, and rename them to c.dll/c.lib, relink and ensure that c.dll is in your path. If in doubt whether the c.dll you have uses TLS, run

dumpbin /EXPORTS c.dll

and look for a .tls entry under the summary section - if it is present it uses TLS and is the wrong library to link with use with excel add-ins.

  ...
  Summary

        4000 .data
        1000 .rdata
        1000 .reloc
        1000 .rsrc
        7000 .text
        1000 .tls

Example

Personal tools
Namespaces
Variants
Actions
Navigation
Print/export
Toolbox