kdb Insights SDK - Release Notes

This page details the release notes for the latest series of kdb Insights SDK releases. To find the release notes for an older release, refer to the previous releases page.

1.14.1

Release Date 2025-07-08

Fixes

This release includes numerous bug, stability and security fixes.

Expand to read details of some important resolved issues here

Database

• Resolved an issue with GUID sorting. Users experienced an EOD mismatch error when there was late data and EOD attempts to sort data across multiple dates.

SQL

• .kxi.sql queries that use EXCEPT ALL have been corrected to align with postgreSQL behaviour, in which duplicates would be included in query results.

Security

• Various CVEs have been patched as part of this release.

Artifacts

KX Nexus

The KX Downloads Portal is the preferred repository for all use cases, and KX Nexus links will be removed once the KX Nexus repository is decommissioned. Nexus links on this page are provided as a temporary alternative to the KX Downloads Portal for existing users.

Stream Processor

service	Nexus Docker	Downloads Portal Docker
Controller	registry.dl.kx.com/kxi-sp-controller:1.14.1	portal.dl.kx.com/kxi-sp-controller:1.14.1
Worker	registry.dl.kx.com/kxi-sp-worker:1.14.1	portal.dl.kx.com/kxi-sp-worker:1.14.1
Coordinator	registry.dl.kx.com/kxi-sp-coordinator:1.14.1	portal.dl.kx.com/kxi-sp-coordinator:1.14.1
Worker-Python	registry.dl.kx.com/kxi-sp-python:1.14.1	portal.dl.kx.com/kxi-sp-python:1.14.1
Worker-ML	registry.dl.kx.com/kxi-ml:1.14.1	portal.dl.kx.com/kxi-ml:1.14.1

Database

service	Nexus Docker	Downloads Portal Docker	Helm Chart
Data Access Process (DA)	registry.dl.kx.com/kxi-da:1.14.1	portal.dl.kx.com/kxi-da:1.14.1
Single container	registry.dl.kx.com/kxi-da-single:1.14.1	portal.dl.kx.com/kxi-da-single:1.14.1
Storage Manager	registry.dl.kx.com/kxi-sm-single:1.14.1	portal.dl.kx.com/kxi-sm-single:1.14.1
Gateway	registry.dl.kx.com/kxi-sg-gw:1.14.1	portal.dl.kx.com/kxi-sg-gw:1.14.1
Resource Coordinator	registry.dl.kx.com/kxi-sg-rc:1.14.1	portal.dl.kx.com/kxi-sg-rc:1.14.1
Aggregator	registry.dl.kx.com/kxi-sg-agg:1.14.1	portal.dl.kx.com/kxi-sg-agg:1.14.1
Rollback	registry.dl.kx.com/kxi-db-rollback:1.14.1	portal.dl.kx.com/kxi-db-rollback:1.14.1
Reference Deployments	N/A	Docker Bundle	N/A
Reference Deployments	N/A	N/A	Kubernetes Bundle

Reliable Transport

service	Nexus Docker	Downloads Portal Docker	Helm Chart
Reliable Transport	registry.dl.kx.com/kxi-rt:1.14.0	portal.dl.kx.com/kxi-rt:1.14.0	kxi-rt-1.14.0.tgz

Miscellaneous

service	Nexus Docker	package
Metrics Sidecar	registry.dl.kx.com/kxi-sidecar:1.14.0
OpenAPI SDK Generator	registry.dl.kx.com/kxi-openapi-codegen:0.8.0	https://nexus.dl.kx.com/repository/kx-insights-packages/kxi-openapi-codegen/0.8.0/kxi-openapi-codegen-0.8.0-shaded.jar

1.14.0

Release Date 2025-06-30

kdb Insights Enterprise 1.14 delivers powerful new language interfaces and improvements (C#, C, Python), greater flexibility for data subscriptions, and many performance and usability improvements. Read on to find out more!

New Features

1. New C# Interface
2. Subscribe to multiple topics of a data stream
3. RT Python Interface Enhancements

1. New C# Interface

A new C# language interface has been added for publishing & subscribing to data in kdb Insights Enterprise from client applications implemented in C#. This language interface supports cross-platform client application deployment on Windows and Linux. Refer to Using the kdb Insights C# interface for details.

2. Subscribe to multiple topics of a data stream

Reliable Transport now allows subscription to multiple topics of the data stream, giving users greater flexibility to only publish/subscribe to messages of interest. This gives greater control over how data is passed through the system and can boost throughput. This release makes this feature available in the Python, Java, C, and q RT interfaces. Note that this will be available in the Stream Processor's UI Reader/Writer in a future release. Refer to Subscribe for details on how to subscribe to a full stream or particular topics within the stream.

3. Reliable Transport Python Interface Enhancements

• The RT Python interface can now be used to subscribe to updates from an RT stream and leverage RT topic filtering. Refer to the Subscribe section of the Python API documentation.

• You can now query kdb Insights Enterprise data using the RT Python interface. Refer to the grant_type parameter and the kxi.query.Query API for how to configure an authenticated connection, and the Quickstart guide to help set this up.

Improvements

This release includes a variety of improvements across the entirety of kdb Insights Enterprise!

Expand to see detail of the full set of improvements!

Python 3.11 Upgrade

• The Stream Processor and Data Access Processes have been upgraded to run Python 3.11, improving performance and providing access to the latest language features.

Potential Breaking Change

Note that this upgrade may impact any User-Defined Functions/Analytics or Python code in a pipeline that was developed against the prior version. Refer to the Important Upgrade & Deployment Considerations section for full details on potential breaking changes.

Data Flow Observability

• A new API for Reliable Transport has been added to help troubleshoot data flow issues where publisher clients have deduplication enabled. Refer to RT deduplication for details of the API and also to the kdb Insights Reliable Transport documentation for more general information on the RT deduplication capabilities.

SQL2

• .kxi.sql2 queries that contain more than one UNION, EXCEPT, INTERSECT operation are now supported for combining queries. For example:

  (select a, b from table1) union (select a, b from table2) except (select a, b from table3)
  

  Note

  When using EXCEPT ALL in SQL2, the result presently aligns to q/kdb+ behavior (that is, all duplicates are removed) rather than postgresql behavior where duplicates would be included. This will be resolved in a future patch.

• SQL2 queries with multiple set operation clauses are now supported.

  ((w union x) intersect y) except z
  

• .kxi.sql2 now supports current_time in the SELECT and WHERE clause of a query.

Improved performance and efficiency of pipeline ingestion from object storage

• SP object store read performance has been significantly enhanced through the introduction of parallelization in downloading and processing files. This improvement greatly reduces load times for large datasets, enabling faster data ingestion and processing from object storage. To take advantage of these improvements, update your pipelines to use the v2 versions of the Amazon S3, Microsoft Azure Storage or Google Cloud Storage readers. Refer to the guide Ingesting from Object Storage for details.

Configurable Timeout on Test Pipeline

• Pipeline testing now includes configurable timeouts for both Quick and Full tests. Users can adjust timeout settings to accommodate specific environmental circumstances regarding data arrival. Refer to Test Timeout for details.

Fixes

This release includes numerous bug, stability and security fixes.

Expand to see detail of some important resolved issues here

Stream Processor

• Resolved an issue related to input ordering for nodes that utilize multiple inputs.

Database

• DBM now resumes a data migration process that was interrupted due to a crash, as opposed to losing progress.

SQL2

• Fixed an issue where an error would be thrown in a SQL2 join query where only the table was referenced, and no SELECT … FROM [table name] clause was provided.

• Fixed an issue where a SQL2 query would error if the column in the WHERE clause did not appear in the SELECT clause.

Security

• Various CVEs were remediated as part of this release.

Important Upgrade and Deployment Considerations

Python 3.11 Upgrade

The upgrade to Python 3.11 in the Stream Processor and Data Access Processes brings the following important changes:

1. EmbedPy Removal: Customer code using EmbedPy no longer works.

2. PyKX Upgrade: PyKX has been upgraded from version 2.5.5 to 3.1, which includes breaking changes to PyKX behavior. For more details, refer to this article.

3. Machine Learning API Update:

   The scikit-learn library has been upgraded from 1.5.2 to 1.6.1. This upgrade introduces the following breaking API changes:

   • adaBoostClassifier: The SP operator no longer supports SAMME.R.
   • randomForestClassifier: The maxFeatures parameter now defaults to sqrt instead of auto.
   • Time-Based Features: Time columns are no longer automatically converted to a supported type (integer or float).

getData

The behavior of distinct when used in the agg parameter of the arguments of .kxi.getData is now always returned enlisted. Refer to the following example:

1) With `agg=enlist`distA`distinct`a` would get the following payload returned:

Old Behavior:

distA
-----
1
0
2
4

New Behavior:

distA
-------
1 0 2 4

2) With `agg=((`distA`distinct`a);(`distB`distinct`b))` both old and new behavior would return this:

distA   distB
-----------------
1 0 2 4 4 0 2 1 3

Artifacts

KX Nexus

The KX Downloads Portal is the preferred repository for all use cases, and KX Nexus links will be removed once the KX Nexus repository is decommissioned. Nexus links on this page are provided as a temporary alternative to the KX Downloads Portal for existing users.

Stream Processor

service	Nexus Docker	Downloads Portal Docker
Controller	registry.dl.kx.com/kxi-sp-controller:1.14.0	portal.dl.kx.com/kxi-sp-controller:1.14.0
Worker	registry.dl.kx.com/kxi-sp-worker:1.14.0	portal.dl.kx.com/kxi-sp-worker:1.14.0
Coordinator	registry.dl.kx.com/kxi-sp-coordinator:1.14.0	portal.dl.kx.com/kxi-sp-coordinator:1.14.0
Worker-Python	registry.dl.kx.com/kxi-sp-python:1.14.0	portal.dl.kx.com/kxi-sp-python:1.14.0
Worker-ML	registry.dl.kx.com/kxi-ml:1.14.0	portal.dl.kx.com/kxi-ml:1.14.0

Database

service	Nexus Docker	Downloads Portal Docker	Helm Chart
Data Access Process (DA)	registry.dl.kx.com/kxi-da:1.14.0	portal.dl.kx.com/kxi-da:1.14.0
Single container	registry.dl.kx.com/kxi-da-single:1.14.0	portal.dl.kx.com/kxi-da-single:1.14.0
Storage Manager	registry.dl.kx.com/kxi-sm-single:1.14.0	portal.dl.kx.com/kxi-sm-single:1.14.0
Gateway	registry.dl.kx.com/kxi-sg-gw:1.14.0	portal.dl.kx.com/kxi-sg-gw:1.14.0
Resource Coordinator	registry.dl.kx.com/kxi-sg-rc:1.14.0	portal.dl.kx.com/kxi-sg-rc:1.14.0
Aggregator	registry.dl.kx.com/kxi-sg-agg:1.14.0	portal.dl.kx.com/kxi-sg-agg:1.14.0
Rollback	registry.dl.kx.com/kxi-db-rollback:1.14.0	portal.dl.kx.com/kxi-db-rollback:1.14.0
Reference Deployments	N/A	Docker Bundle	N/A
Reference Deployments	N/A	N/A	Kubernetes Bundle

Reliable Transport

service	Nexus Docker	Downloads Portal Docker	Helm Chart
Reliable Transport	registry.dl.kx.com/kxi-rt:1.14.0	portal.dl.kx.com/kxi-rt:1.14.0	kxi-rt-1.14.0.tgz

Miscellaneous

service	Nexus Docker	package
Metrics Sidecar	registry.dl.kx.com/kxi-sidecar:1.13.0
OpenAPI SDK Generator	registry.dl.kx.com/kxi-openapi-codegen:0.8.0	https://nexus.dl.kx.com/repository/kx-insights-packages/kxi-openapi-codegen/0.8.0/kxi-openapi-codegen-0.8.0-shaded.jar