Skip to content

Deploying the ICE Consolidated Feed Handler with kdb Insights Enterprise

The following section details how to install and run the KX ICE Consolidated Feed Handler on kdb Insights Enterprise.

Architecture

KX ICE FH Architecture - Enterprise

Prerequisites

Before running the KX ICE Consolidated Feed Handler on kdb Insights Enterprise, ensure the following prerequisites are met:

Setting up Kubernetes Secrets

First, create a Kubernetes secret containing your ICE login credentials:

kubectl create secret generic my-ice-secrets --from-literal=my-username=<insert ice username> --from-literal=my-password=<insert ice password> -n <insert kubernetes namespace>

Then, create a secret containing your credentials for the KX Docker registry portal.dl.kx.com :

kubectl create secret -n <insert kubernetes namespace> docker-registry regcred --docker-username=<insert registry username> --docker-password=<insert registry password> --docker-server=portal.dl.kx.com

Downloading the Feed Handler Helm Chart

The Helm chart for the KX ICE Consolidated Feed Handler is available on KX's public chart repository. Add this repository to your local Helm configuration by running the following:

helm repo add kx-ext https://portal.dl.kx.com/assets/helm --username <insert portal username> --password <insert portal password>

Update the repository to load the latest available charts:

helm repo update kx-ext

Use helm search to list the available versions of the KX ICE Consolidated Feed Handler:

helm search repo kx-ext/rt-ice-pub

Select the latest version of the chart from the list of versions, then download it:

helm fetch kx-ext/rt-ice-pub --version  <insert latest version> --untar --devel

Note

Replace <insert latest version> with the latest version of the rt-ice-pub chart, for example 1.0.1.

This downloads and untars the rt-ice-pub chart in your current working directory.

Configuring the Feed Handler

The default configuration for the KX ICE Consolidated Feed Handler is in the values.yaml file. To modify this configuration, create a copy:

cp rt-ice-pub/values.yaml my-values.yaml

Edit my-values.yaml to configure the Feed Handler.

Key sections in this configuration file include:

ice

Contains the settings for configuring the KX ICE Consolidated Feed Handler. For detailed information of each setting, refer to the ICE Configuration documentation. Ensure the following settings are correct:

  • primaryConnection: Verify the ICE primary connection host/ports.
  • backupConnection: Verify the ICE backup connection host/ports.
  • secrets: Ensure the name of the ICE credentials secret matches the one created in Setting up Kubernetes Secrets. Leave the username and password settings as is.

rt

In a kdb Insights Enterprise deployment, typically the KX ICE Consolidated Feed Handler publishes data to an RT stream. This section configures the interaction between an RT stream and the KX ICE Consolidated Feed Handler. For detailed information of each setting, refer to the RT documentation. Ensure the following setting is correct:

  • sinkName: Set this to the name of the RT stream for publishing. For example, set it to rt-ice-example-package-ice-north for the example package discussed later.

imagePullSecrets

The imagePullSecrets section is for listing the names of secrets required by the KX ICE Consolidated Feed Handler. The names of both secrets created in Setting up Kubernetes Secrets should be displayed here:

imagePullSecrets:
  - name: regcreds
  - name: my-ice-secrets

Deploying a kdb Insights Enterprise App

To get started with ingesting and querying ICE market data, download the example kdb Insights Enterprise package by running the following:

version=1.0.1;\
curl -L -O https://portal.dl.kx.com/assets/raw/kxi-ice-fh/${version}/ice-example-package-${version}.kxi

Use this package as the base to build your own new package, or take elements from it and integrate them into an existing package.

To deploy the package run the following set of commands:

user@demo-host:~/ice-example$ ls
ice-example-package-1.0.1.kxi
user@demo-host:~/ice-example$
user@demo-host:~/ice-example$ kxi package push ice-example-package/1.0.1
Pushing: ice-example-package-1.0.1.kxi
{
    "ice-example-package": [
        {
            "version": "1.0.1",
            "_status": "InstallationStatus.SUCCESS"
        }
    ]
}
user@demo-host:~/ice-example$
user@demo-host:~/ice-example$ kxi package deploy ice-example-package/1.0.1
user@demo-host:~/ice-example$

Note

This example kxi package expects the default KX ICE Consolidated Feed Handler configuration. Only the configuration fields discussed in Configuring the Feed Handler should be changed.

Deploying the Feed Handler

Deploy the KX ICE Consolidated Feed Handler using the my-values.yaml file created and edited earlier.

!!! Reminder For the feed handler to publish successfully to the ice-example-package, myvalues.yaml must have sinkName: rt-ice-example-package-ice-north under the stream: section.

user@demo-host:~/helm_charts/ice$ ls
my-values.yaml  rt-ice-pub
user@demo-host:~/helm_charts/ice$
user@demo-host:~/helm_charts/ice$ helm install rt-ice-pub rt-ice-pub -f my-values.yaml -n <insert namespace>
NAME: rt-ice-pub
LAST DEPLOYED: Tue Jun 11 14:58:55 2024
NAMESPACE: user
STATUS: deployed
REVISION: 1
TEST SUITE: None
user@demo-host:~/helm_charts/ice$

Querying the Data

Once the ice-example-package and KX ICE Consolidated Feed Handler have been deployed successfully, navigate to the kdb Insights Enterprise UI and query the data.

kdb Insights Enterprise ICE Market Data Query

Viewing the Feed Handler Logs

View the KX ICE Consolidated Feed Handler logs using kubectl:

kubectl logs rt-ice-pub-0 | less

Teardown the Feed Handler

To teardown (uninstall) the KX ICE Consolidated Feed Handler, run:

user@demo-host:~/helm_charts/ice$ helm uninstall rt-ice-pub -n <insert namespace>
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/lbritton/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/lbritton/.kube/config
release "rt-ice-pub" uninstalled
user@demo-host:~/helm_charts/ice$

Teardown the Example Package

To teardown the ice-example-package, run: 

kxi package teardown ice-example-package/1.0.1

To delete all data associated with the package, run:

kxi package teardown --rm-data ice-example-package/1.0.1

Warning

The --rm-data flag erases all data in the specified package's database. Use with caution.