QforMortals3/Websockets 101

From Kx Wiki
Jump to: navigation, search

1.21 Websockets 101

In traditional web applications, the browser (as client) initiates requests and the server replies with the page or data requested using the http protocol. The web server does the serious data manipulation. In recent years, browsers and Javascript have evolved to levels of sophistication that permit quite powerful processing to be done safely on the client side—for example, input editing and display formatting. Indeed, you can use WebSockets to put a browser front end on traditional applications, replacing both the web server and proprietary GUI packages in one fell swoop (phrase used with its original meaning).

The key idea of WebSockets is that the client makes an initial http request to upgrade the protocol. Assuming the request is accepted, subsequent communication occurs over TCP/IP sockets protocol. In particular, once the WebSockets connection is made, either the client or server can initiate messaging.

We begin with a simple example that demonstrates how to connect a browser to a q process via WebSockets, request data and then display the result. We assume familiarity with basic html5 and Javascript. Enter the script below in a text editor and save it as sample1.html in a location accessible to your browser.

Information.png Note: This script uses the c.js script for serialization and deserialization between q and javascript, which you can download from http://kx.com/q/c/c.js. For simplicity in this demo, we have placed a copy in the same directory as the sample1.html script so that it can be loaded with a (trivial) relative path. You should point this to the location of c.js in your q/kdb installation.

This script creates a minimal page with a field to display the result returned from the q server. After declaring some useful variables, we get to the interesting bits that create the WebSockets connection and handle the data from q. We create a WebSockets object and set its binaryType property to “arraybuffer”, which is necessary for the exchange of q serialized data.

We wire the behavior of the connection object by attaching handler functions for WebSockets events. Most important are then onopen and onmessage events. When the connection is opened, we serialize a Javascript object containing a payload string and send it (asynchronously) to the q process; it will arrive there as a serialized dictionary. Conversely, when a serialized message is received from the q process, we deserialize it and invoke the sayN function on the content. The sayN function locates the display field on the page and copies its parameter there.

<!doctype html>
<html>
<head>
<script src="c.js"></script>
<script>
 var serverurl = "//localhost:5042/",
     c = connect(),
     ws;

  function connect() {
    if ("WebSocket" in window) {
      ws = new WebSocket("ws:" + serverurl);
      ws.binaryType="arraybuffer";
      ws.onopen=function(e){ 
        ws.send(serialize({ payload: "What is the meaning of life?" }));
      };
      ws.onclose=function(e){ 
      };
      ws.onmessage=function(e){
        sayN(deserialize(e.data));
      };
      ws.onerror=function(e) {window.alert("WS Error") };
    } else alert("WebSockets not supported on your browser.");
  }

	function sayN(n) {
		document.getElementById('answer').textContent = n;
	}
</script>

Now to the server side, where the q code is blissfully short. Start a fresh q session, open port 5042 and set the web socket handler .z.ws to a function that will be called on receipt of each message from the browser.

q)\p 5042
q).z.ws:{0N!-9!x; neg[.z.w] -8!42}

The handler first displays its parameter, which we do for demonstration only as we have no further use for it in this example. Then it serializes the answer using (-8!) and sends it back (asynchronously) to the browser. That’s it!

Now point your browser to

file:// sample1.html

and you should see the answer displayed in a font large enough to be seen from the international space station. Notice that there is no web server here other than q itself.

Now that we’re warmed up, let’s try a more interesting example. Here the browser will ask the q server to call out to the Yahoo Finance site and retrieve historical stock price and volume data for Apple. The browser will then use the publicly available High Charts package to display a spiffy stock history graph. In this example there will be a bit more Javascript and q, but it’s tolerable. And again, no web server other than q.

First let’s tackle the browser side. Save the following text file as ws101.html.

<!DOCTYPE HTML>
<html>
<head>
  <script src= 
    "http://ajax.googleapis.com/ajax/libs/jquery/1.8.2/jquery.min.js">
  </script>
  <script src="http://code.highcharts.com/stock/highstock.js"></script>
  <script src=   
    "http://code.highcharts.com/stock/modules/exporting.js"></script>
</head>
<body>
 <div id="container" style="height: 500px; min-width: 500px"></div>
</body>
<script src="c.js"></script>
<script>
  var serverurl = "//localhost:5042/",
     ws,
     c = connect(),
     ticker = "AAPL",
     startdt = "2008-01-01";
 
  function connect() {
    if ("WebSocket" in window) {
      ws=new WebSocket("ws:" + serverurl);
      ws.binaryType="arraybuffer";
      ws.onopen=function(e){ 
        toQ([ticker, startdt]);
      };
      ws.onclose=function(e){ 
      };
      ws.onmessage=function(e){
        return fromQ(e.data);
      };
      ws.onerror=function(e) {window.alert("WS Error") };
    } else alert("WebSockets not supported on your browser.");
  }

  function toQ(pl) {
    ws.send(serialize({ payload: pl }));
  }
  function fromQ(raw) {
    var data = deserialize(raw);
    return createChart(data); 
  }
  function toUTC(x) {
    var y = new Date(x); 
    y.setMinutes(y.getMinutes() + y.getTimezoneOffset()); 
    return y.getTime();
  }
  function createChart(data) {
    $('#container').highcharts('StockChart', {
      rangeSelector : {
        selected : 1
      },
      title : {
        text : ticker + ' Stock Price'
      },		
      series : [{
        name : ticker,
        data : data.hist.map(function(x){return [toUTC(x.Date), x.Close]}),
        tooltip: {
          valueDecimals: 2
        }
      }]
    });
  }
</script>
</html>

We provide a concise description of the highlights, assuming familiarity with basic html5 and Javascript.


Information.png Tip: The alignment between Javascript objects and q dictionaries, along with built-in serialization/deserialization, makes data transfer between the browser and q effortless.

Now to the q server side, which has fewer lines of code that do much work. Save the following text file as ws101.q.

\p 5042

adjDate:{[dt] 0 -1 0i+`year`mm`dd$dt};     / 2015.01.01 -> 2015 0 1i

getHist:{[ticker; sdt; edt]
  tmpl:"\http://ichart.finance.yahoo.com/table.csv?s=",
        "%tick&d=%em&e=%ed&f=%ey&g=d&a=%sm&b=%sd&c=%sy";
  args:string ticker,raze adjDate each (sdt;edt);
  url:ssr/[tmpl; ("%tick";"%sy";"%sm";"%sd";"%ey";"%em";"%ed"); args];
  raw:system "wget  -q -O - ",url;
  t:("DFFFFJF"; enlist ",") 0: raw;
 `Date xasc `Date`Open`High`Low`Close`Volume`AdjClose xcol t}

getData:{[ticker; sdt] select Date,Close from getHist[`$ticker;"D"$sdt;.z.D]}
         
.z.ws:{
  args:(-9!x) `payload;
  neg[.z.w] -8!(enlist `hist)!enlist .[getData; args; `err]}

Following is a description of the q code.

The final entry in the q script sets the web sockets handler. Here is its behavior.

To run this example, start a fresh q process and load the ws101.q script.

q) \l /pages/ws101.q		

Now point your browser at

file:///pages/ws101.html

Wait a few seconds (depending on the speed of your internet connection) and—Voila!—you have a nifty graph of AAPL stock price.



Prev: Example: Asynchronous Callbacks, Next: Overview

Reprinted with the author's permission from: q for Mortals Version 3, An Introduction to Q Programming by Jeffry A. Borror.

The book is available on Amazon. In the United Kingdom, it is available at Amazon UK.

©2015 Jeffry A. Borror/ q4m LLC

Personal tools
Namespaces
Variants
Actions
Navigation
Print/export
Toolbox