kdb Insights SDK - Release Notes

This page details the release notes for the latest series of kdb Insights SDK releases. To find the release notes for an older release, refer to the previous releases page.

1.18.0

Release date: 2026-03-11

kdb Insights SDK 1.18 introduces enhancements focused on developer productivity and operational efficiency. This release provides greater stream filtering capability, expands querying capabilities, and strengthens reliability, performance, and operational control across the platform.

New Features

Stream Topic Filtering

Pipeline developers can now publish or subscribe to specific data streams with greater control. This allows pipelines to get more targeted with their data of interest and reduce the overhead of processing data they do not need. This reduces resources requirements and overall cost of managing all the data that is is being by distributed by RT streams.

Version 2 of the Stream Reader and Writer provide the following additional features:

The v2 reader supports server side topic filtering.
Allows multiple Stream Readers and Writers in the same pipeline to point to different Streams.
Stream-side filtering of messages.
In the web interface, v2 nodes allow you to configure the stream directly within the node, rather than through the Settings tab.

Refer to Streams in Pipelines for an example.

Improvements

Checkpointing in Data Writedown

To improve efficiency and reliability, the database now logs a checkpoint after each table is written to disk. If an End-of-Day (EOD) writedown is interrupted after some tables or dates have been processed, a retry resumes from the last completed checkpoint and preserves the data already written. Previously, the entire EOD writedown would restart from the beginning. This is enabled by default, but can be disabled by setting KXI_SM_EOD_CHECKPOINT to false.

Querying

When calling an API or UDA, the query now automatically routes to the DAPs(s) and Aggregator(s) for which they are registered, without the need to specify the scope.assembly parameter.
Moved some Data Access Process initialization logging from DEBUG to TRACE to avoid unnecessary logging.
Support for the following SQL Operators has been added:
- NOW
- NULLIF
- COALESCE
- CASE
- IS NULL and IS NOT NULL
[BETA] A new optional key can be set at the system level, or added to query request options, to enable tracking of the execution time of queries. This tracks and reports:
- which DAPs are participating in the request, as well as the RC/AC codes and the duration of the execution.
- which Aggregator is responsible for aggregating, as well as the duration of aggregation.
- which RCs are participating in the request (primary and secondaries) including processing and queuing time.
Refer to Timing UDA Functions for an example using a user-defined analytic for demonstration.
An option has been added to API calls to be able to choose the aggregator that the query is directed to. Refer to Request Header for details.

Note

Use caution with this feature and clearly understand your data model and storage configuration. This is most applicable to development or debugging activity. For example, where you may want to intentionally work with a given AGG where you have loaded code or debug/break statements etc.

Database

Added support of offset for the IDB as a means of reducing resource contention. This allows you to stagger EOIs if you have multiple Storage Managers on the same node with the same frequency. Refer to the offset parameter for details.

Stream Processor

The MQTT reader is not compatible with stream or database writers that have deduplication enabled. This restriction prevents potential data loss from MQTT during recovery scenarios. Previously, a validation issue in the web interface allowed this unsupported configuration. This issue has been fixed, and such pipelines will no longer run. If you need to run this type of pipeline and accept the associated data loss risks, disable deduplication on the writer by setting the KXI_ALLOW_NONDETERMINISM environment variable.

Reliable Transport

Caching changes have been made to help reduce disk I/O usage by Reliable Transport. RT can now cache message headers and optionally cache the message payload. Refer to Sub-server Caching for details.
Resiliency improvements have been made for power failure scenarios that can cause corruption of RT log files. This reduces the possibility of data flow interruption if there is a loss of power to the RT client and server nodes.

Fixes

RT
- When the configured maximum percentage threshold for archival is reached, RT logs are now truncated correctly.
Querying
- Fixed an issue where if the first line of qSQL was a comment, the query failed.
- All UDA queries now properly honor the timeout parameter.
- If DAPs or Aggregators are not connected to a Resource Connector, a warning is now displayed.
Database
- Data migration between HDB tiers with different compression settings no longer fails to apply a sorted attribute on a symbol column.
- Batch ingest now correctly recognizes new symbols when executed immediately after an EOI.
- Fixed an issue during a schema conversion where validation was skipped when multiple conversions were combined, causing the conversion to fail.
- Fixed an issue where execution of queries was prevented after certain malformed queries were passed to .kxi.selectTable.
- Terminating a Storage Manager process no longer results in unnecessary errors being logged by the DAPs.
- Fixed an issue where the timeout for downloading of packages by the Storage Manager and DAPs was set to 30 seconds and not the configurable timeout value.
- Fixed an issue where when hdr.cast was set to true, distinguished arguments (table,inputTZ,scope, etc.) would not get cast.
- Fixed an issue where the Storage Manager (SM) did not reject schemas containing a column whose oldName property matched an existing column name. The oldName property must now be removed if it remains from a previous column rename.

Artifacts

KX Nexus

The KX Downloads Portal is the preferred repository for all use cases, and KX Nexus links will be removed once the KX Nexus repository is decommissioned. Nexus links on this page are provided as a temporary alternative to the KX Downloads Portal for existing users.

Stream Processor

service	Nexus Docker	Downloads Portal Docker
Controller	registry.dl.kx.com/kxi-sp-controller:1.18.0	portal.dl.kx.com/kxi-sp-controller:1.18.0
Worker	registry.dl.kx.com/kxi-sp-worker:1.18.0	portal.dl.kx.com/kxi-sp-worker:1.18.0
Coordinator	registry.dl.kx.com/kxi-sp-coordinator:1.18.0	portal.dl.kx.com/kxi-sp-coordinator:1.18.0
Worker-Python	registry.dl.kx.com/kxi-sp-python:1.18.0	portal.dl.kx.com/kxi-sp-python:1.18.0
Worker-ML	registry.dl.kx.com/kxi-ml:1.18.0	portal.dl.kx.com/kxi-ml:1.18.0

Database

service	Nexus Docker	Downloads Portal Docker	Helm Chart
Data Access Process (DAP)	registry.dl.kx.com/kxi-da:1.18.0	portal.dl.kx.com/kxi-da:1.18.0
Single container	registry.dl.kx.com/kxi-da-single:1.18.0	portal.dl.kx.com/kxi-da-single:1.18.0
Storage Manager	registry.dl.kx.com/kxi-sm-single:1.18.0	portal.dl.kx.com/kxi-sm-single:1.18.0
Gateway	registry.dl.kx.com/kxi-sg-gw:1.18.0	portal.dl.kx.com/kxi-sg-gw:1.18.0
Resource Coordinator	registry.dl.kx.com/kxi-sg-rc:1.18.0	portal.dl.kx.com/kxi-sg-rc:1.18.0
Aggregator	registry.dl.kx.com/kxi-sg-agg:1.18.0	portal.dl.kx.com/kxi-sg-agg:1.18.0
Rollback	registry.dl.kx.com/kxi-db-rollback:1.18.0	portal.dl.kx.com/kxi-db-rollback:1.18.0
Reference Deployments	N/A	Docker Bundle	N/A
Reference Deployments	N/A	N/A	Kubernetes Bundle

Reliable Transport

service	Nexus Docker	Downloads Portal Docker	Helm Chart
Reliable Transport	registry.dl.kx.com/kxi-rt:1.18.0	portal.dl.kx.com/kxi-rt:1.18.0	kxi-rt-1.18.0.tgz

Miscellaneous

service	Nexus Docker	package
Metrics Sidecar	registry.dl.kx.com/kxi-sidecar:1.18.0
OpenAPI SDK Generator	registry.dl.kx.com/kxi-openapi-codegen:0.8.0	https://nexus.dl.kx.com/repository/kx-insights-packages/kxi-openapi-codegen/0.8.0/kxi-openapi-codegen-0.8.0-shaded.jar