Skip to content

Deploying the ICE Consolidated Feed Handler with kdb Insights SDK

The following section details how to install and run the KX ICE Consolidated Feed Handler using the kdb Insights SDK.

Architecture

KX ICE FH Architecture - Insights SDK

Pre-Requisites

Before running the KX ICE Consolidated Feed Handler with the kdb Insights SDK, the following prerequisites must be met:

  • docker is installed.
  • docker-compose is installed.
  • You have login credentials for the ICE Consolidated Feed.
  • You have login credentials for docker registry at portal.dl.kx.com.
  • You have a kdb Insights license file (kc.lic).

Login to Docker Registry

To access the images needed for this walkthrough, login to the KX Docker registry:

docker login portal.dl.kx.com -u <insert registry username> -p <insert registry password>

Download the ICE kdb Insights SDK Example

To download the ICE kdb Insights SDK example, run:

version=1.0.1;\
curl -L  https://portal.dl.kx.com/assets/raw/kxi-ice-fh/${version}/ice-insights-sdk-example-${version}.tar.gz -o ice-insights-sdk-example.tgz

Extract (untar) the files and navigate to the resulting directory:

tar -xvzf ice-insights-sdk-example.tgz;\
cd ice-insights-sdk-example

Configure the ICE kdb Insights SDK Example

Within the ice-insights-sdk-example directory, locate the docker-compose-one-node.yaml file and the env file. Update the following in the env file:

  • Update the kx_licence_dir variable to the directory path where your kdb Insights license file is located.
  • Update the KXFEED_ICE_USERNAME and KXFEED_ICE_PASSWORD variables with your ICE username and password.

File Permissions

To avoid permission issues with containers mounting directories, run the following within the ice-insights-sdk-example directory. This command may require sudo access:

chmod -R 777 */

Or, use sudo if necessary:

sudo chmod -R 777 */

Deploy the ICE kdb Insights SDK Example

After updating the configuration, deploy the ICE kdb Insights SDK example:

docker-compose --env-file env -f docker-compose-one-node.yaml up

Querying the Data

After deploying, open a second terminal to query the Quote data published by the ICE feed using this getData REST call:

START=$(date "+%Y.%m.%dD00:00:00.000000000");\
END=$(date "+%Y.%m.%dD23:59:59.999999999");\
curl -X POST "localhost:8080/kxi/getData" -H "Content-Type: application/json" -H "Accepted: application/json" --data "{\"table\":\"Quote\",\"startTS\":\"$START\",\"endTS\":\"$END\"}"

To query the Trade data, use:

START=$(date "+%Y.%m.%dD00:00:00.000000000");\
END=$(date "+%Y.%m.%dD23:59:59.999999999");\
curl -X POST "localhost:8080/kxi/getData" -H "Content-Type: application/json" -H "Accepted: application/json" --data "{\"table\":\"Trade\",\"startTS\":\"$START\",\"endTS\":\"$END\"}"

If neither call returns any data, review the logs to check for any issues.

Viewing the Logs

To check if the services have started successfully, view the logs using the following command within the ice-insights-sdk-example directory:

docker-compose --env-file env -f docker-compose-one-node.yaml logs | less

If the ICE feed is publishing successfully, logs from the sp worker are displayed with messages such as, Received quote or Received trade.

If these messages are absent, search for ERROR and FATAL messages to identify any issues. If no errors are found, validate that it is market hours for the symbols listed in the KXFEED_ICE_SUBSCRIPTION_LIST setting in the env file.

Note

Logs also appear when you run docker-compose up, but they can move quickly and be difficult to read. Use docker-compose logs for easier reading and searching.

Tearing Down the ICE kdb Insights SDK Example

To teardown and delete the containers for the ICE kdb Insights SDK example, run the following command within the ice-insights-sdk-example directory:

docker-compose --env-file env -f docker-compose-one-node.yaml down

To remove all data written to disk by the example, first delete all volumes:

docker volume rm docker_compose_rt-north-0;\
docker volume rm docker_compose_rt-north-session;\
docker volume rm docker_compose_rt-south-0;\
docker volume rm docker_compose_rt-south-session;

Then, remove all data written to mounted paths (requires sudo access):

sudo bash -c 'rm -rf db/data/db/*; rm -rf db/data/logs/*; rm -rf log/*; rm -rf sp/checkpoints/*'