kdb+ 2.8 Release Notes¶

This page details the latest features and improvements released to kdb+ 2.8.

2015.05.28¶

Fixes and improvements¶

a SIGINT/'stop or wsfull during .Q.gc[] when using slave threads could cause a crash.

2014.11.26¶

Fixes and improvements¶

in,bin was erroneously sensitive to sort attr with compound types. e.g.
```
q)(10b;10b)~([] a:0 1; b:`)in/:(([] a:0 0; b:`);([] a:`s#0 0; b:`))
```
find could crash for ()?int vector. e.g.
```
q)()?til 99
```

2014.05.03¶

Fixes and improvements¶

aggregating partial expressions would crash across partitioned databases. e.g.
```
q)`:p/0/t/`:p/1/t/ set\:([]a:til 10);system"l p";select sum(a+)from t
```

2014.03.19¶

Fixes and improvements¶

searching on compound key was erroneous since last release. e.g.

q)K:([a:1 2;b:`x`y]c:5 6);d:`a`b`c!(3;`c;4);K,:d;0N!K~([a:1 2 3;b:`x`y`c]c:5 6 4);

2014.03.16¶

Fixes and improvements¶

improve performance of searching on compound key e.g. x:([]a:..)cross([]b:..)

q)n:1000000;d:.z.d;t1:([]date:d;a:n?`4;b:n?1000f;c:n?1000000);m:100000;t2:([date:d;a:m?`4];b:m?1000f;d:m?1000)
q)0N!system"ts t1 lj update date:`#date from t2";system"ts t1 lj update date:`s#date from t2"

2014.01.24¶

Fixes and improvements¶

match of mapped nested vectors was incorrect. e.g.

q)`:t/ set([]x:(1 2;3 4);y:(1 2;5 6));t:get`:t/;t[`x]~t`y

2013.11.20¶

Fixes and improvements¶

ij could perform very poorly when lhs had >99 rows, ignoring any attributes on 1^st col of rhs

e.g. q)a:([]a:til 1;b:til 1);b:`a`b xkey update `g#a,c:i from([]a:til 1000)cross([]b:til 1000);(100#a)ij b

-3!x now more efficient for stringify of enums
```
e.g. q)i:50000000?0Wi;e:`t!i;system"ts -3!e"
```
allow "P"$string .z.d so as to be consistent with "Z"$string .z.d.

2013.09.05¶

Fixes and improvements¶

update with scalars for linked columns was not retaining type. Root cause was catenate.

e.g. q)t:([]1 2 3);([]x:1 2 3;p:`t!3#0N)~update p:`t!0N from t
     q)((enlist`x)!enlist 1 2 3),(enlist`p)!`t!enlist 0N

assigning to .Q.s could cause a crash
```
e.g. q).Q.s:{}
```

2013.06.20¶

Fixes and improvements¶

writing a list of primitives to a file could corrupt the heap, causing a crash at some point later, since v1.0
```
e.g. q)`:test set 3000000#(::);.Q.gc[]
```

2013.06.14¶

New¶

heap accounting for reading compressed files is now based on what data is actually read rather than the full vector length,
```
and hence -w limits are less likely to be hit.
```

Fixes and improvements¶

using keyed tables as a step function was throwing 'nyi since 2013.06.06
```
e.g. q)(`s#([sym:1#`a;date:1#.z.d-1]symout:1#`a))flip(`a;100#.z.d);
```

2013.06.06¶

Fixes and improvements¶

w32 only - alloc > 4GB caused crash instead of reporting wsful.
```
e.g. q)1000000000#1
```
nested sync msgs could cause the socket to be closed erroneously, the remote throwing 'rcv: Resource temporarily unavailable.
```
e.g. q)h:hopen 55557;neg[h]@/:("0N!.z.w \"0N!{0}[]\"";"0N!.z.w \"0N!{1}[]\"")
```
N.B. nested sync messaging is generally to be avoided; due to the nature of having to process incoming messages to reach a response,
```
the order of response messages may be incorrect.
```
multicolumn find (t1?t2) was slow for some scenarios if t1 had an attribute on the first column.

2013.03.05¶

Fixes and improvements¶

within inside select clause was not using comparison tolerance.

e.g. q)v:1+(til 3)*1e-16;0N!3=count select from ([]v) where v within v 2 2;

garbage collect could result in seg fault instead of printing wsfull.
```
e.g. q)((1+)\)0 / iterate forever on 32bit kdb+
```

2013.01.04¶

Fixes and improvements¶

comparison tolerance was too tolerant on 0=1e-256 (0 should only be equal to 0).

2012.12.22¶

Fixes and improvements¶

upsert with missing columns of nested types did not fill using those types (used () instead).
```
e.g. q)t:([]a:2 3;b:("as";"qwe"));t,:([]a:3 4);([]a:2 3 3 4;b:("as";"qwe";"";""))~t
```

2012.12.20¶

Fixes and improvements¶

` vs `sym?`a.b`c was crashing, instead of throwing 'type.

2012.12.05¶

New¶

sv and vs now support enums on rhs

e.g. q)s:`sym?`a.b;0N!`a`b~` vs s;s:`sym?`a`v;0N!`a.v~` sv s;

Fixes and improvements¶

lsq was crashing if passed empty lists; now throws 'length
```
e.g. q)(enlist 0#0f)lsq(0#0f;0#0f)
```

2012.09.06¶

Fixes and improvements¶

select with a "date.[month|year] in .." was not matching correctly if rhs was not sorted and unique.
```
e.g. q)select from([]t:enlist 2001.01.01)where t.month in 2001.02 2001.01m
```
setting a `p attribute on a vector which already had a `u attribute was crashing (immediately or later), since v1.0
```
e.g. group `p#`u#-20?20;
```

2012.09.02¶

Fixes and improvements¶

added rank check to lj to catch error case which was causing a crash
```
e.g. q)([]a:1 2;d:1 2),\:([a:(1 2;1 2)]f:3 4)
```
q.k - .Q.bv kept files in the last partition mapped if they were empty.

2012.08.02¶

Fixes and improvements¶

built-in logging via -l could log bad data if incoming ipc msg was compressed.

2012.07.12¶

Fixes and improvements¶

a splay is defined as a special form of dict. dict,splay was crashing.
```
e.g.(`a`b!`a`d),`a`!`
```

2012.06.22¶

Fixes and improvements¶

within was throwing 'type when comparing .z.z/.z.p with time of days

q)select from ([] times:`s#(.z.z;.z.z+1;.z.z+2)) where times within 10:00 11:00

2012.06.06¶

Fixes and improvements¶

setting a variable name with a : in it was being interpreted as error trap, since v2.8 2012.02.14.
```
e.g. q).[`$"b:1";();:;2]
```

2012.05.29¶

Fixes and improvements¶

q.k - cor function was returning values > 1.
```
e.g. q)1 -1 1 cor 1 -1 0N
```
patch for v2.8 to use ipc with v3.0, to allow it to send timestamp/timespan.

2012.05.10¶

Fixes and improvements¶

closing stdin pipe from parent process could send kdb+ into a spin reading stdin

2012.04.20¶

Fixes and improvements¶

dict _ symbol was crashing since 2012.04.18

2012.04.18¶

Fixes and improvements¶

table _ symbol was crashing
```
q)([]())_`
```
hclose .z.w within .z.ph was leaving outgoing data on the .z.w handle.

2012.04.03¶

Fixes and improvements¶

.Q.gc[] could crash when used with slave threads, since v2.7 2011.09.21.
an aborted incoming connection could result in the server process stopping listening for new connections ('accp error).
catch failure to init .z.a on startup (hostname lookup fails, .z.a assigned -1).
functional select was crashing when "order" argument did not evaluate to an int vector.
```
e.g. q)t:([]a:`a`b;b:0 1.);?[t;();enlist[`a]!enlist`a;();2;`b]
```

2012.03.21¶

Fixes and improvements¶

Appending to a file of size <=8 bytes was sometimes failing with 'type

q)@[`:.;`foo;:;()];f:{@[`:.;`foo;,;1 2 3];};f[]; / sometimes failed depending on what was on native stack

2012.03.08¶

Fixes and improvements¶

Attempting an update to a partitioned table would crash; now blocked

q)`:/tmp/db/2001.01.01/t/ set([]a:til 10);`:/tmp/db/2001.01.01/t/
q)\l /tmp/db
q)@[`t;0;{x+y+z}[1];2]

upsert on a table with a group attribute on a column was slow (previous attempt was kdb+2.7 2011.11.12)
```
e.g. q)t:([k:`u#til 2000000]v:`g#2000000?1000000);system"t `t upsert([]k:count[t]+til 4;v:4#-1)"
```
block x!y if either one is splayed, as {x!x}flip`a`b!`c was crashing

passing bad args to file compression could crash, e.g.

q)a:"abracadabra";b:`a`q`y`x`s`e;f:{1+x*x};m:4 5@20?1000
q)(a;b;f;m) set `:test4

2012.03.01¶

amended entry for 2011.10.06 regarding compression of # files

2012.02.20¶

Fixes and improvements¶

1: parsing as booleans was erroneous since 2011.11.22
```
e.g. ("bbb";1 1 1)1:0x000102
```

2012.02.14¶

(updated)

Fixes and improvements¶

allow error trapping of `::5000 data
```
e.g. q)@[`::5000;"a+b";0N!]
```
.z.pc was not firing if a timeout occurred in the process whilst it was awaiting a response to a sync request.
code submitted as text via ipc was not identifying comments if they followed a \n, since 2.8 2011.10.06
```
q)raze -4! "/ Comment1\n{};\n/ Comment2"
```

2012.02.05¶

New¶

allow single shot sync request as `:host:port:user:password data
```
q)`:localhost:5000 "2+2"
```

2012.02.02¶

Fixes and improvements¶

peach was crashing on Solaris only since 2.7 2011.09.14

Selecting a splay table from itself caused either a wsful or crash

q)system"cd /tmp";system"mkdir t";system"cd t";`:2012.01.01/t/ set([]foo:1 2);system"l .";
q)select t,1 from t

2012.01.20¶

New¶

Optionally allow tables to be missing from partitions, by scanning partitions for missing tables and taking the

tables' prototypes from the last partition. After loading/re-loading from the filesystem, invoke .Q.bv[]
to (re)populate .Q.vt/.Q.vp, which are used inside .Q.p1 during the partitioned select .Q.ps.

Fixes and improvements¶

Block -ve number of slave threads as a cmd line arg; now throws 'bad slave arg
Invoking gc in combination with a single slave thread (probably a useless configuration) was crashing.
A connection could be dropped if a msg was pending on a socket, and processing due to another socket caused those msgs to be read; since v1.0.

2012.01.04¶

Fixes and improvements¶

delete key from dict was crashing

q)d:`x`y`z!(100 200 300);delete x from `d;

2011.12.31¶

Fixes and improvements¶

hopen timeout was sometimes not working since 2.7 2011.05.15

2011.12.15¶

Fixes and improvements¶

compressing empty files was resulting in a file that could not be read - arises with nested vectors with a single distinct value.
```
e.g. q)(`:fooz;17;2;9) set 1000#enlist"";get`:foo
```

2011.12.13¶

Fixes and improvements¶

q.k - redid uj. Was inconsitent with types returned when nested lists were involved.
```
e.g. q)a:([id:1 2 3 4]a:4 5 6 8);b:([id:1 3]b:(4 6;6 7 8));0N!a uj b;0N!b uj a
```
delete from `t still had some instability, sometimes crashing, since 2011.11.18

was crashing if wrong types passed to -19!x, also via set

e.g. q)(`:splay/;``a((17 2 9);(17 2 5)))set([]a:1000?10) / missing ! in dict definition

2011.12.02¶

Fixes and improvements¶

updates from within .z.ts when in multithreaded input mode are now threadsafe on windows; they were already safe on other platforms.
update c:`a`b!1 2 from ([]c:()) was crashing

2011.11.29¶

Fixes and improvements¶

append to a compressed file containing a nested vector could corrupt that file causing wsful or seg fault when next read.
```
e.g. q)t:([]c:2#enlist 10?.Q.a);(`:test/;17;2;9)set t;`:test/ upsert t;-8!get`:test
```

2011.11.28¶

Fixes and improvements¶

system command now uses unique temporary filenames (unique across processes)

file compression - algorithm 0 (no compression) now writes out raw non-compressed data, with no block/meta info.

allow a dict of filenames!zipParams to be passed with set or -19!x. The ` entry in the dict is the default zipParams,
 or if no ` specified then default will be do not compress.
e.g. q)(`:splay/;``a`b!((17;2;9);(17;2;6);(17;2;6)))set([]a:asc 1000000?10;b:asc 1000000?10;c:asc 1000000?10)

Appending to a compressed file on windows could corrupt the file.

2011.11.22¶

Fixes and improvements¶

delete from `t was crashing since 2011.11.18
```
e.g. t:([]a:();b:());delete b from `t
```
1: non-zeros being parsed as booleans are now interpreted as 1b.
```
e.g. ("bbb";1 1 1)1:0x000102
```
q.k - .Q.k now set to 2.8 to avoid previous versions loading it.

2011.11.21¶

First production release of kdb+ 2.8.

Fixes and improvements¶

foreign key lookups for dictionaries and time function lookups, were crashing (on solaris sparc only) since 2.7 2010.12.06
```
e.g. t:([a:`a`b]b:3 4);d:`x`y!(`t$`a;12:34);exec x.b,y.hh from d
```

2011.11.18¶

Fixes and improvements¶

delete c from`whatever now takes care of views; previously it was crashing.
```
e.g. q)value"a.c::a.b";delete c from `a;a.b:3
```

In some cases, deleting from a list would leave an empty of the wrong type instead of ().

e.g. q)s:flip `a`b!"**"$\:();do[2;`s upsert `a`b!("bar";`a`b!1 2)];do[2;s:s _ 0];`s upsert `a`b!("bar";`a`b!1 2)

better detect loop dependencies

q)a.c::a
q)delete c from `. / was crashing

file compression enabled for windows builds.

2011.11.17¶

Fixes and improvements¶

appending to compressed files corrupted the files.

2011.11.16¶

Fixes and improvements¶

append to dictionary could leave the dictionary as an invalid object (causing a crash later) if the append failed.
```
e.g. q)d:`a`b!2 3;d[`c]:0b
```
time+minute or time+seconds was throwing 'type
```
e.g. q).z.t+08:00
```
select with an empty list of columns caused a crash.
```
e.g. q)select()from
```
reading a partial timestamp (e.g. "02.10.2011T04:")in a csv file could cause a crash.
keyed tables with a column of dictionaries was throwing type on upsert
```
e.g. q)t:([a:enlist"bar"]b:enlist `a`b!1 2);`t upsert 1#t
```
raze of types from 100...106 would throw 'type.
```
e.g. q)raze (1;+;3 4)
```
block Q) and \l file.Q as win/osx filenames are not case sensitive (continuously reloads q.k without defining Q).
```
e.g. Q)2+2
```

increased limits for glim (number of vectors with a `g# attr) and mlim (number of concurrently mapped nested vectors) to 65530.

e.g. q){`g#x}each enlist each til 65530;
     q)`:t/ set([]x:enlist til 1);{x set select from get`:t/;}each `$"i",/:string til 65530;

.Q.gc[] could cause instability/crash when invoked, since 2011.09.15
decompressing a file that was compressed with ipc algo could result in a crash.
upsert was failing for keyed tables where key only was specified; value extends with ().
```
e.g. q)t:([()]y:());`t upsert([]1 2)
```
()in/:() was returning `boolean$() but should have been ()
block v[i]+:t

q.k - redfine cov:{avg(x-avg x)*y-avg y}

e.g. q)100000000 100000001 100000002j cov 100000000 100000001 100000002j
    q)v:10#0;v[0 1]+:([]a:til 2)

allow < comparison of ordinal (.z.z and .z.p) and cardinal types. Converts (take day portion) and truncates to the cardinal type.
```
e.g. q).z.z<21:00
```

g attribute was ignored when reapplying it

e.g. q)v:`g#2000000?1000000;system"t v:`g#v"

q.k - redfine rand:{*1?x} , system:."\\" and faster j10,j12

upsert on a table with a `g# on a column was slow since v2.5

e.g. q)t:([k:`u#til 2000000]v:`g#2000000?1000000);system"t `t upsert([]k:count[t]+til 4;v:4#-1)"

plist' with more than 8 args was crashing. Now throws 'length if more than 8 args.
```
e.g. q)plist'[1;1;1;1;1;1;1;1;1;1;1]
```

2 operations on nested arrays were causing a crash

e.g. q)`:nest set (0 1 2;3 4 5);get[`:nest]_1
     q)=':[get`:nest]

2011.10.06¶

New¶

q.k - set now supports (`:file|`:splay/;blockSize;algo;zipLevel) for streaming compression.

append to (kdb+ built-in compression) compressed files

e.g. q)(`:zippedTest;17;2;9) set 100000?10;`:zippedTest upsert 100000?10;-21!`:zippedTest

'compress new files' mode - active if .z.zd (zip defaults) present and valid.

.z.zd can be set to an int vector of (blockSize;algo;zipLevel) to apply to all new files which do not have a file extension.
-19!x and (`:file;size;algo;level) set x override .z.zd for that file/splay

NUC (Not Upwardly Compatible)¶

Not backward compatible. Changes existing behavior.

-19!(`:infile;`:outfile;blockSize;algo;zipLevel) now returns outfile. Previously it returned achieved compression. Use -21! to obtain meta info.
-19!x will not revoke compression if compressed size > original size; this is to avoid to and fro with append to compressed files.

changed parser to properly allow (/) etc, supporting each right as standalone,

e.g. q)type(/:)
[space]/ is a comment but ;/ (/ [/ {/ are not. No one should be using these as comments.
Please do a grep on your code to see if you ever used these.

On unix, system redirects output for capture via /tmp/XXXXXX
```
e.g. q)system"ls"
```

File compression - don't try to compress a # file for nested data, as it will be compressed

as part of compressing its partner file. e.g.
q)`:a set 1000#enlist asc 1000?10;-19!(`:a;`:za;17;2;9);0N!get[`:a]~get`:za;

Fixes and improvements¶

improved mmu performance (osx,linux)
```
e.g. q)\t x$flip x:380 70000#1.0
```
osx build now supports multithreaded modes