Ingestion examples overview

This section details some common use-cases for ingesting data into kdb Insights Enterprise using the kdb Insights CLI.

KX Nexus

The KX Nexus repository will be sunsetted in the future. Nexus links on this page are provided as a temporary alternative to the KX Downloads Portal for existing users. The KX Downloads Portal is the preferred repository for all use cases and KX Nexus links will be removed once the KX Nexus repository is decommissioned.

• Reading from Kafka
• Batch ingest from S3 buckets
• Publishing from the language interfaces

Downloading assemblies

Each of the examples above (not being generated by the UI) deploys a unique assembly CR. The files used by these samples are hosted on the KX Downloads Portal and should be downloaded to run locally. If you deployed from Azure Marketplace, reach out to Support and they can provide sample assembly examples for you.

Existing users can use the KX Nexus repository until Nexus is removed in the future. You can retrieve the same sample files from the KX Nexus repository and use them in the same way as described above.

Firstly, get the correct version of the assemblies by running the command below.

export INSIGHTS_OPVERSION=$(kubectl get deployment/insights-kxi-operator --namespace kxi-operator -o jsonpath='{.metadata.labels.app\.kubernetes\.io/version}')

The assembly files can be downloaded and applied. The code block below does this for the RT client assembly using the kdb Insights CLI.

Replace <USER> and <PASSWORD> in the example command below with your Nexus credentials.

wget --user=<USER> --password=<PASSWORD> https://portal.dl.kx.com/assets/raw/assembly-samples/${INSIGHTS_OPVERSION}/sdk_sample_assembly.yaml

kxi assembly deploy --filepath sdk_sample_assembly.yaml

Querying data

After data is ingested by one of the sample assemblies, it can be queried using the Service Gateway.

For full details on the available functionality for querying, refer to Data query overview.