Skip to content

kdb Insights SDK - Release Notes

This page details the release notes for the latest series of kdb Insights SDK releases. To find the release notes for an older release, refer to the previous releases page.

1.17.0

Release Date 2025-12-12

kdb Insights SDK 1.17 delivers a wide variety of improvements across the system. Read on to find out more!

Improvements

This release includes a variety of improvements across the entirety of kdb Insights SDK.

Expand to see the details of the full set of improvements.

Stream Processor

  • autoRegister support has been added for Avro schemas using the Kafka writer. See the autoRegister parameter of the Kafka writer for details.

Database

  • When using multiple labels in a query, you can now set the environment variable KXI_ALLOW_PARTIAL_LABELS to allow partial results to be returned in the event that the relevant data for each label does not fully intersect. This will return the data for the label combinations that do exist, and will also provide information on which combinations could not return data. See the KXI_ALLOW_PARTIAL_LABELS parameter for details.

  • Added the ability to use date as the groupBy column in getData queries. See the groupBy parameter for getData for details.

  • A subset of a schema's tables can now be imported via Initial Import. Any tables that did not yet have data will be created empty at startup, following the database validation operation, allowing for future data ingestion. https://kxl.atlassian.net/browse/KXI-63483

  • When defining a User-Defined Analytic, the scope field can now be set as mandatory.

  • Added a new helper function kxi.appendLabels for User-Defined Analytics creation, which takes table data and adds the current DAPs assembly labels to the table.

  • The database now supports changing a column from scalar to compound

RT Java SDK

  • The ability to pass arguments to a DataAccessClient's ping method has been added, allowing you to ping a particular assembly. See ping for details.

Fixes

This release includes numerous bug, stability and security fixes.

Expand to read details of some important resolved issues here.

Database

  • Fixed an issue in the Service Gateway where subscription requests sent to the client might not receive a SUBNAP response.

  • Changed resource coordinator default replicas to 1 to align with the number of default DAPs.

  • There is now protection against accidentally assigning globals in .kxi.qsql. A side-effect of this is that queries such that make use of tableName.column type notation will no longer work. These can be fixed by changing to tableName`column. For example, min trade.price is fixed by switching to min trade`price.

  • Resolved an issue with the KXI_SM_EOI_REQUESTS_PENDING metric, which was incorrectly displaying negative values after emergency EOIs, even though there was no actual processing problem.

  • It is no longer possible to batch ingest data where the date of prtnCol differed from the defined partition date. If this is attempted, the batch ingest will be rejected entirely.

  • When a schema is changed, the old schema will be honoured for any data added to RT logs before the database restart and schema conversion. Publishers must not begin publishing in the new schema before the database is restarted to perform the schema change; otherwise, a schema mismatch error will occur.

  • Improved the error handling of database migration to object storage. Previously, in some cases, if the upload failed, data loss was possible.

  • Batch ingest now explicitly rejects incoming data for the time range overlapping with object storage tier. Previously, the Storage Manager discarded such data.

  • Added a validation rejecting non-partitioned data provided to batch ingest in a date partition. Previously, such data was erroneously written to HDB partition.

  • When EOD writedown discards data older than the configured retention period, SM now prints a warning message.

  • Fixed an issue where, with KXI_LATE_DATA disabled, DAP purviews could be incorrect after SM restart, causing query timeouts.

Stream Processor

  • Upgraded librdkafka version from 1.9.2 to 2.11.1.

  • The SP coordinator /create API has been updated to accept tolerations. See the SP Coordinator OpenAPI spec for details.

Security

  • Security patches have been applied and CVEs have been remediated.

Important Upgrade and Deployment Considerations

Default Configurations

The default number of Resource Coordinators (RC) deployed has been changed to 1. This is to align to the default setting of 1 for Data Access Processes (DAPs). In general, there needs to be as many (or more) DAPs as there are RCs. See Ordinal Connection for details.

Database

It is recommended to have an inventory file in place when one of your database tiers resides on object storage to help speed up reload times. Previously, the file information was configured by setting the KX_OBJSTR_INVENTORY_FILE environment variable. This is no longer supported, as both SM and DAPs now handle this by themselves. Refer to Object storage inventory files for more details.

If you have an inventory file in place, it will be necessary to change the configuration of its location reference. In your package file, you will now need to specify the location of the inventory file using the location parameter of the sm.tiers[].inventory property.

This sample configuration illustrates an S3 tier defined in a package file:

- name: s3
    mount: hb
    store: s3://kxi-sm-example/db
    inventory:
     enabled: true
     location: [file location here]

Note

The DAP will no longer start if the environment variable is set directly.

Artifacts

KX Nexus

The KX Downloads Portal is the preferred repository for all use cases, and KX Nexus links will be removed once the KX Nexus repository is decommissioned. Nexus links on this page are provided as a temporary alternative to the KX Downloads Portal for existing users.

Stream Processor

service Nexus Docker Downloads Portal Docker
Controller registry.dl.kx.com/kxi-sp-controller:1.17.0 portal.dl.kx.com/kxi-sp-controller:1.17.0
Worker registry.dl.kx.com/kxi-sp-worker:1.17.0 portal.dl.kx.com/kxi-sp-worker:1.17.0
Coordinator registry.dl.kx.com/kxi-sp-coordinator:1.17.0 portal.dl.kx.com/kxi-sp-coordinator:1.17.0
Worker-Python registry.dl.kx.com/kxi-sp-python:1.17.0 portal.dl.kx.com/kxi-sp-python:1.17.0
Worker-ML registry.dl.kx.com/kxi-ml:1.17.0 portal.dl.kx.com/kxi-ml:1.17.0

Database

service Nexus Docker Downloads Portal Docker Helm Chart
Data Access Process (DAP) registry.dl.kx.com/kxi-da:1.17.0 portal.dl.kx.com/kxi-da:1.17.0
Single container registry.dl.kx.com/kxi-da-single:1.17.0 portal.dl.kx.com/kxi-da-single:1.17.0
Storage Manager registry.dl.kx.com/kxi-sm-single:1.17.0 portal.dl.kx.com/kxi-sm-single:1.17.0
Gateway registry.dl.kx.com/kxi-sg-gw:1.17.0 portal.dl.kx.com/kxi-sg-gw:1.17.0
Resource Coordinator registry.dl.kx.com/kxi-sg-rc:1.17.0 portal.dl.kx.com/kxi-sg-rc:1.17.0
Aggregator registry.dl.kx.com/kxi-sg-agg:1.17.0 portal.dl.kx.com/kxi-sg-agg:1.17.0
Rollback registry.dl.kx.com/kxi-db-rollback:1.17.0 portal.dl.kx.com/kxi-db-rollback:1.17.0
Reference Deployments N/A Docker Bundle N/A
Reference Deployments N/A N/A Kubernetes Bundle

Reliable Transport

service Nexus Docker Downloads Portal Docker Helm Chart
Reliable Transport registry.dl.kx.com/kxi-rt:1.17.0 portal.dl.kx.com/kxi-rt:1.17.0 kxi-rt-1.17.0.tgz

Miscellaneous

service Nexus Docker package
Metrics Sidecar registry.dl.kx.com/kxi-sidecar:1.17.0
OpenAPI SDK Generator registry.dl.kx.com/kxi-openapi-codegen:0.8.0 https://nexus.dl.kx.com/repository/kx-insights-packages/kxi-openapi-codegen/0.8.0/kxi-openapi-codegen-0.8.0-shaded.jar