Cookbook/Websocket

From Kx Wiki
Jump to: navigation, search

The wiki is moving to a new format and this page is no longer maintained. You can find the new page at code.kx.com/q/cookbook/websockets.

The wiki will remain in place until the migration is complete. If you prefer the wiki to the new format, please tell the Librarian why.

Contents

Simple websocket server example

kdb+v3.0 supports the websocket protocol.

To get your browser and kdb+ talking websockets, start a q session listening on localhost:5000

q -p 5000

Get ws.htm, a simple websocket client. Save this HTML file and open it in a websocket capable browser. You should see something like this:

Wso.png

Now click connect and type something like 4+til 3 in the edit box. hit enter or click send. Note that it is echoed in the output text area.

Wsecho.png

How it works

Kdb+ serves all types of protocols on the same port and the websocket protocol is no exception. .z.ws is called for every message sent from the client (browser). The default behaviour is {neg[.z.w]x} which echoes the message back to the client - so kdb+ is an echoing websocket server when you start it listening.

Doing something useful

We just need to define .z.ws to do something useful. In your q session at your console, define:

.z.ws:{neg[.z.w].Q.s value x}

Then try typing the same text in the edit box 4+til 3 and submit. You will see a result this time:

Ws.png

To enable error-reporting, try:

.z.ws:{neg[.z.w]@[.Q.s value@;x;{"'",x,"\n"}]}

c.js (no AJAX required)

c.js (also at http://kx.com/q/c/c.js) provides functions serialize and deserialize to simplify IPC between the browser and a kdb+ server. An example, wslogin.htm shows how to send a javascript dictionary to kdb+. It receives an echo of the dictionary and turns it back into a javascript dictionary.

To run this example

A byte vector is passed to .z.ws when using the c.js function serialize. To decode a serialized string, use -9! and to encode, use -8!

 .z.ws:{neg[.z.w] -8!value -9!x;}

Note that the above functions for .z.ws won't work when using serialize/deserialize. To handle both byte and char, check for the type of the input.

This example works, because the default .z.ws echoes the byte vector over the Websocket.

JSON

Arthur wrote a small utility to parse/generate json. Since v3.2 2014.08.22 it is shipped in q.k. For prior releases, it can be found at http://kx.com/q/e/json.k

After Loading json.k, example data in .j.x
q).j.x
C    b j z                      
--------------------------------
"as" 0                          
""   1 2 2014.05.08T16:45:16.025
q).j.j .j.x / convert q data to json
"[{\"C\":\"as\",\"b\":false,\"j\":null,\"z\":\"\"},{\"C\":\"\",\"b\":true,\"j\":2,\"z\":\"2014-05-08T16:45:16.025\"}]"
q)-1 .j.j .j.x; / pretty display
[{"C":"as","b":false,"j":null,"z":""},{"C":"","b":true,"j":2,"z":"2014-05-08T16:45:16.025"}]
q).j.k .j.j .j.x / convert json to q
C    b j z                        
----------------------------------
"as" 0   ""                       
""   1 2 "2014-05-08T16:45:16.025"

n.b. if your json data is spread over multiple lines, reduce those to a single char vector with raze. e.g.

$ cat t1.json 
{
	"code" : 3,
	"message" : "This request requires authorization"
}
q).j.k raze read0`:t1.json
code   | 3f
message| "This request requires authorization"

Otherwise you'll encounter an error similar to
q).j.k read0`:t1.json
k){=\~("\\"=-1_q,x)<q=x}
'length
<
(0b;,0b;000000000000b;00000000000000000000000000000000000000000000000000b)
(,0b;010000100000b;01000000010001000000000000000000000000000000000001b;,0b)
q.j))\\

Compression

v3.2t 2014.05.08 added 'permessage-deflate' websockets compression. One way to observe whether compression is active on a connection is to observe the queued data. For example

/ generate some compressible data
q)v:10000#.Q.a 
/ queue 1000 msgs to an existing websocket handle
/ and observe the queue
q)\ts do[1000;(-5)v];show sum each .z.W
5| 10004000
6| 0
14 20610976
/ now do same again, but this time with a handle which requested compression
q)\ts do[1000;(-6)v];show sum each .z.W
5| 0
6| 47022
94 4354944

Here we can see the noncompressed data was quicker to add to the queue, and consumed more memory overall. The compressed data took longer to queue, and in this case was 200x smaller. Compression speed and ratio achieved will depend on your data.

In Chrome you can also observe the network handshake in view->developer->developer tools; a succesful negotiation will have "Sec-WebSocket-Extensions:permessage-deflate" in the http response header.

Simple websocket client example

Since V3.2t 2014.07.26, kdb+ can also create a websocket connection, i.e. operate as a client in addition to as a server.

.z.ws must be defined before opening a websocket

q).z.ws:{0N!x;} / print incoming msgs to console, no echo.

A websocket can be created with the syntax

 q)r:(`$":ws://host:port")"GET / HTTP/1.1\r\nHost: host:port\r\n\r\n"

If successful it will return a 2 element list of (handle;http response), e.g.

(3i;"HTTP/1.1 101 Switching Protocols\r\nConnection: Upgrade\r\nUpgrade: websocket\r\nSec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk=\r\nSec-WebSocket-Extensions: permessage-deflate\r\n\r\n")

and from that point on will callback via .z.ws when msgs arrive. To send msgs, use

q)neg[handle]"text" / a char vector
or
q)neg[handle]0x010203 / a bytevector

If the protocol upgrade from http to websocket failed, it returns the 2 element list, with the handle as 0Ni, e.g.

(0Ni;"HTTP/1.1 400 Bad Request\r\nContent-Type: text/html; charset=UTF-8...")

The response text is returned for debug purposes only; ideally, you need only be concerned whether the handle is valid.

Any other error is thrown as usual, e.g.

 'www.nonexist.badcom: No route to host

Should you need to use websockets over ssl, e.g. wss://host:port, consider stunnel, and open from kdb+ to that stunnel with ws://. Basic Authentication can be passed in the char vector on opening, along with any other necessary fields such as cookies etc.

Both client and server support permessage-deflate compression.

Secure sockets: stunnel

http://en.wikipedia.org/wiki/Stunnel will provide secure sockets (TLS/SSL) using the OpenSSL library. Stunnel will take any websocket server, HTTP server, or similar and secure it - you get https:// and wss:// for free.

Also https://github.com/cesanta/ssl_wrapper

UTF8 Encoding

The websocket requires that text is UTF8 encoded. If you try to send invalidly encoded text it will throw 'utf8.

See also

Personal tools
Namespaces
Variants
Actions
Navigation
Print/export
Toolbox