Skip to content

Adding custom logic to kdb Insights Enterprise

The introduction and use of custom code within kdb Insights Enterprise from external sources is facilitated in its entirety through the use of packages and the kdb Insights CLI. Packages within the context of the kdb Insights Enterprise are a standardized, flexible format which provide a mechanism by which your versioned code can be introduced to a running instance of the kdb Insights Enterprise. Once uploaded to kdb Insights Enterprise, these packages are stored internally to your deployment on a shared volume. This allows the code to be used simultaneously by the Stream Processor (Pipeline), Scratchpad and Database for analytic development.

Once in the system, interactions with your code are driven principally via a Python and q interface. These interfaces provide mechanisms to load custom Python/q code from versioned packages. They also provide a mechanism by which you can define application-specific User Defined Functions(UDFs) for deployment to the Stream Processor. The Python and q APIs provided as part of your kdb Insights Enterprise deployment can also be used locally for the development

To get started with the creation of packages and the introduction of custom code to your system, follow the quickstart guide outlined here. On completion of this end-to-end guide, you will understand how to:

  1. Create a new package
  2. Add custom code and UDFs to the package
  3. Create an uploadable artifact
  4. Introduce a new package to a kdb Insights Enterprise instance
  5. Make use of this package within the Stream Processor, Scratchpads and Database

Steps 1-3 are covered within the "Creating a package" section, step 4 is discussed in "Uploading a package" and the use of packages within the various supported locations is covered in "Using a package".

For a full breakdown on the APIs which you can use when creating, uploading and interacting with packages please use the following links:

Installation instructions/requirements

To generate and interact with packages you must first install the kdb Insights CLI following instructions here. You can then create packages, upload them to kdb Insights Enterprise and interact with packages that already exist on your instance.

To interact with packages locally, in order to facilitate iterative development, you can install q or Python, which allows you to load packages and UDFs within local q and Python sessions respectively.