Skip to content

Quickstart

Two workflows with the kxi-ml-analytics package:

  1. Integrate it with your own qpk Docker image
  2. Run it standalone on premises

Download kxi-ml-analytics.qpk

Package as a dependency

Set the environment variables required for generation of the qpk Docker image.

kxi-ml-analytics requires the base image centos:8

export QPDOCKER_BASE=centos:8

Write requirements.txt in the root of your application:

numpy
scipy
scikit-learn<=0.23.0
statsmodels
matplotlib
sobol-seq
pandas>=1.0

Generate a qp.json to make the package a dependency of your application. It must contain both

  • the kxi-ml-analytics package
  • py to pick up the necessary Python requirements as dependencies
├── init.q
├── qp.json
├── requirements.txt
└── deps/
    └── kxi-ml-analytics.qpk

qp.json:

{
  "myapp" : {
    "name"    : "my-application",
    "depends" : [ "py", "deps/kxi-ml-analytics" ],
    "entry"   : [ "init.q" ]
  }
}

Create the Docker image for your application:

qp build myapp

Run the application using either QPacker or Docker:

$ qp run myapp
q).ml.clust
             | ::
i            | ``df`lf`dists`closest`reIndex`floatConversion`err`checkK`check..
kd           | ``i`newTree`nn`q`findleaf`qC`c`findLeaf!(::;``tree`nnCheck`fin..
kmeans       | ``fit`predict`update!(::;{[data;df;k;config]
  data:clust.i.fl..
$ source qpbuild/.env
$ docker run -it -e "KDB_LICENSE_B64=$KDB_LICENSE_B64" $myapp
q).ml.clust
             | ::
i            | ``df`lf`dists`closest`reIndex`floatConversion`err`checkK`check..
kd           | ``i`newTree`nn`q`findleaf`qC`c`findLeaf!(::;``tree`nnCheck`fin..
kmeans       | ``fit`predict`update!(::;{[data;df;k;config]
  data:clust.i.fl..

Standalone on premises

This library does not depend on Docker, nor on external packages that do, so it can be consumed as a standalone library:

All QPKs are ZIPs and can be unpacked

unzip kxi-ml-analytics.qpk

Check the requirements are installed. To use this application on-prem you need embedPy installed locally and the Python requirements:

pip install -r kxi-ml-analytics/requirements.txt

Initialize the library from the unzipped directory using the startq.q script:

$ cd kxi-ml-analytics
$ q startq.q
q).ml.clust
             | ::
    i            | ``df`lf`dists`closest`reIndex`floatConversion`err`checkK`check..
    kd           | ``i`newTree`nn`q`findleaf`qC`c`findLeaf!(::;``tree`nnCheck`fin..
    kmeans       | ``fit`predict`update!(::;{[data;df;k;config]
      data:clust.i.fl..

Define and use a loading function in an initialization script.

├── init.q
└── deps/
    ├── kxi-ml-analytics/
    └── kxi-ml-analytics.qpk

init.q:

// Define the location from which your dependecies are to be loaded
PACKAGE_PATH:{[path]$[count path;path;"deps"]}[getenv`PACKAGE_PATH]
// Define the loading function
loadfunc:{[pkg]
  pwd:system"cd";
  system"cd ",PACKAGE_PATH;
  if[not(`$pkg)in key`:.;
    system"cd ",pwd;
    '"unable to locate package: ",pkg
    ];
  system"cd ",pkg;
  err:@[{system"l ",x;::};"startq.q";::];
  system "cd ",pwd;
  if[10h=type err;'"Failed to load package: ",err]
  }
// Load the unzipped qpk
@[loadfunc;"kxi-ml-analytics";{'"Failed to load package with error: ",x}]
$ q init.q
q).ml.clust
             | ::
    i            | ``df`lf`dists`closest`reIndex`floatConversion`err`checkK`check..
    kd           | ``i`newTree`nn`q`findleaf`qC`c`findLeaf!(::;``tree`nnCheck`fin..
    kmeans       | ``fit`predict`update!(::;{[data;df;k;config]
      data:clust.i.fl..