Skip to content

Using the kdb Insights C SDK

The C SDK allows you to publish data to RT from your own application.

This section details how to get started with the C SDK. For examples on how to use it to publish data into a kdb Insights RT Microservice or kdb Insights Enterprise see sample program.

Downloading the C SDK

You can download the C SDK from the kdb Insights Nexus registry.

The kxi-c-sdk-${VERSION}.zip file contains everything required to use the C SDK. No installation is required, you can just download the zip file and unzip.

Supported Operating Systems

The driver is supported on the following operating systems:

  • CentOS 8 or later
  • Red Hat Enterprise Linux (RHEL) 8 or later
  • Ubuntu 18.04 or later

Note

The SDKs are only supported on Linux running on x86 architectures and Windows. We currently do not support OsX.

Files extracted

├── include
│   └── rt_helper
│       ├── kdb
│       │   ├── k.h
│       │   └── kk.h
│       ├── ksvc_direct_c.h
│       ├── ksvcrt_c.h
│       └── rt_params.h
├── librt_helper.so
├── push_client
├── replicator_static.so
└── rt_helper_app

Prerequisites

Before building a program that uses the C SDK you need:

  • C SDK
  • cmake package
  • libssl package
  • libcurl package
  • libcsv package

Using the C SDK in your own application

To publish data to kdb Insights Enterprise using the C SDK as part of your own application, use the rt_helper. This reads all the information needed to connect to kdb Insights Enterprise from the variables provided and publishes the data into the system.

Do one of the following:

  • Extract the details from the sample program
  • Follow the sections below that describe how to publish data

Sample illustration

An example usage of the C SDK to publish to the sdk_sample_assembly is as follows:

Pre-requisites

  • Download the assembly sdk_sample_assembly following the instructions here.
  • Make sure that the assembly is deployed in your kdb Insights Enterprise instance.
  • Ensure you have an authenticated kdb Insights Enterprise client URL.
  • Make sure that the kdb Insights Enterprise ingest endpoints (as defined by the KXI_CONFIG_URL) are accessible.

Initializing

  1. Start the rt_helper, passing the KXI_CONFIG_URL and logging level as parameters. A list of all parameters is available here.

  2. Define the schema. For the sdk_sample_assembly you must define the trace table schema as "sensorID:int,captureTS:ts,readTS:ts,valFloat:float,qual:byte,alarm:byte".

#include <rt_helper/kdb/kk.h>
#include <rt_helper/ksvcrt_c.h>
#include <rt_helper/rt_params.h>

RT_Params params = {.configUrl = "{KXI_CONFIG_URL}", .logLevel = "info"};
void *h = ksvcrtc_start(&params);      

//Define the schema
Schema *schema = ParseSchema("sensorID:int,captureTS:ts,readTS:ts,valFloat:float,qual:byte,alarm:byte");
Parameters

The following parameters are available:

parameter required default description
configUrl Mandatory none The URL or file that this program calls or reads to find the endpoint(s) it needs to connect to
configMaxAge Optional 300000 Maximum age of configuration details in milliseconds. After this amount of time, if still unable to fetch a new configuration, the connection is considered to be broken and any subsequent attempts to send data to kdb Insights Enterprise will fail with a Not connected error. This is only needed when using the kdb Insights Enterprise Information Service to obtain the configuration details.
consoleLogLevel Optional err Console log level (info/warn/err/off)
fetchConfigSleep Optional 5000 Time in ms to sleep between reads of the configuration details
localPersistencePeriod Optional 600000 Local persistence period (in milliseconds)
logLevel Optional info Log level (info/warn/err/off)
rtDir Optional /tmp/rt RT directory
queryTimeout Optional 2000 Milliseconds to wait for the connection to be established

configUrl

The configUrl parameter can be set to: - KXI_CONFIG_URL to access kdb Insights Enterprise from outside the cluster - file:///locaton/config_file_name to use a local file to access kdb Insights Enterprise or a kdb Insights RT Microservice from inside the cluster

​

Handling changing hostnames

Each publisher has a session name associated with it, to ensure it is uniquely identifiable by kdb Insights. This is physically represented within the C SDK as the directory messages are written to before being replicated into kdb Insights.

The C SDK sets the session name (and hence the message directory) to the following: SDK_CLIENT_UID.streamid.hostname. This ensures that publishers on different hosts, that are sending different data to the same kdb Insights Stream, can be correctly identified.

SDK_CLIENT_UID

SDK_CLIENT_UID is part of KXI_CONFIG_URL (https://\({INSIGHTS_HOSTNAME}/informationservice/details/\)) - These can be obtained following the steps here outside the cluster

Currently it is not recommended to use the C SDK in a scenario where the hostname changes between each instantiation (for example, inside a Docker container that is destroyed and re-created between each run) as a new session will be created for each instantiation.

Publishing data

To publish data:

  1. Create the message
  2. Publish the message

The example below takes a csv record and converts it to a kdb+ object before publishing:

//Create a dummy record
char csv_record[] = "10,2021.01.01D00:00:09.000000000,2021.01.01D00:00:09.000000000,97,1,0";

//Convert the CSV record to a kdb+ data type (this function is available in the sample program)
K k_record = convert_csv_record_to_k_record(schema, csv_record);

//Publish the message 
ksvcrtc_insert(h, "trace", k_record);

Terminating

Once you have finished publishing messages you must stop the rt_helper:

//Stop the rt_helper
ksvcrtc_stop(h);