Skip to content

Publishing to kdb Insights Enterprise using q

In this example we will work through a q process sending data to an instance of kdb Insights Enterprise using a 1 or 3 node deployment of RT. The publisher will be running outside of the kdb Insights Enterprise deployment, meaning you could be sending data to kdb Insights Enterprise using a q session running on your laptop.

Reliable Transport (RT) comes with a set of q functions to send or receive message to and from RT. Details on these q function are touched on below, but for more information see here

Authentication

Before data can be sent to kdb Insights Enterprise, a set of authentication steps need to be followed.

These steps are outlined on the kdb Insights Enterprise client enrolment page.

Publish

In this example, we will make use of a RT publish function, .rt.pub. This is an API, maintained by KX, that can be called to send data into RT.

In order to access the API, download the rt.qpk.

version=1.10.0
curl -L  https://portal.dl.kx.com/assets/raw/rt/$version/rt.$version.qpk -o rt.qpk
unzip rt.qpk

Start a q session and publish the data 

$ cd rt
$ q startq.q
q)
q)setenv[`KXI_CONFIG_URL] "https://my-deploy.com/informationservice/details/d955a31fc4deb880e53704794671bd49"; // Note, this is taken from the client enrolment step above
q)params:`config_url`path!(getenv`KXI_CONFIG_URL; "/tmp/rt/");
q)params
config_url| "https://my-deploy.com/informationservice/details/d955a31fc4deb880e53704794671bd49"  
path      | "/tmp/rt/"               // location where the user wishes to write the RT log file. A location with suitable disk space should be chosen.
q)p:.rt.pub[params];            // initialize your connection to RT. This creates a foreign object used to publish.
q)data:([] col1:`a`b`c; col2:1 2 3)
q)p(`.b; `table; data)  // publish to RT

Data format

RT is data format agnostic. The 3 item list we are publishing above is a messageType, table name and payload. The kdb Insights Database and Stream Processor expect data in this format.

Querying Data

You have now sent data into kdb Insights Enterprise. There are a number of ways of querying that data back, including the REST call getData and SQL.