kdb Insights SDK - Release Notes
This page details the release notes for the latest series of kdb Insights SDK releases. To find the release notes for an older release, refer to the previous releases page.
1.14.0
Release Date 2025-06-30
kdb Insights Enterprise 1.14 delivers powerful new language interfaces and improvements (C#, C, Python), greater flexibility for data subscriptions, and many performance and usability improvements. Read on to find out more!
New Features
1. New C# Interface
A new C# language interface has been added for publishing & subscribing to data in kdb Insights Enterprise from client applications implemented in C#. This language interface supports cross-platform client application deployment on Windows and Linux. Refer to Using the kdb Insights C# interface for details.
2. Subscribe to multiple topics of a data stream
Reliable Transport now allows subscription to multiple topics of the data stream, giving users greater flexibility to only publish/subscribe to messages of interest. This gives greater control over how data is passed through the system and can boost throughput. This release makes this feature available in the Python, Java, C, and q RT interfaces. Note that this will be available in the Stream Processor's UI Reader/Writer in a future release. Refer to Subscribe for details on how to subscribe to a full stream or particular topics within the stream.
3. Reliable Transport Python Interface Enhancements
-
The RT Python interface can now be used to subscribe to updates from an RT stream and leverage RT topic filtering. Refer to the Subscribe section of the Python API documentation.
-
You can now query kdb Insights Enterprise data using the RT Python interface. Refer to the
grant_type
parameter and thekxi.query.Query
API for how to configure an authenticated connection, and the Quickstart guide to help set this up.
Improvements
This release includes a variety of improvements across the entirety of kdb Insights Enterprise!
Expand to see detail of the full set of improvements!
Python 3.11 Upgrade
- The Stream Processor and Data Access Processes have been upgraded to run Python 3.11, improving performance and providing access to the latest language features.
Potential Breaking Change
Note that this upgrade may impact any User-Defined Functions/Analytics or Python code in a pipeline that was developed against the prior version. Refer to the Important Upgrade & Deployment Considerations section for full details on potential breaking changes.
Data Flow Observability
- A new API for Reliable Transport has been added to help troubleshoot data flow issues where publisher clients have deduplication enabled. Refer to RT deduplication for details of the API and also to the kdb Insights Reliable Transport documentation for more general information on the RT deduplication capabilities.
SQL2
-
.kxi.sql2
queries that contain more than oneUNION
,EXCEPT
,INTERSECT
operation are now supported for combining queries. For example:(select a, b from table1) union (select a, b from table2) except (select a, b from table3)
Note
When using
EXCEPT ALL
in SQL2, the result presently aligns to q/kdb+ behavior (that is, all duplicates are removed) rather than postgresql behavior where duplicates would be included. This will be resolved in a future patch. -
SQL2 queries with multiple set operation clauses are now supported.
((w union x) intersect y) except z
-
.kxi.sql2
now supportscurrent_time
in theSELECT
andWHERE
clause of a query.
Improved performance and efficiency of pipeline ingestion from object storage
- SP object store read performance has been significantly enhanced through the introduction of parallelization in downloading and processing files. This improvement greatly reduces load times for large datasets, enabling faster data ingestion and processing from object storage. To take advantage of these improvements, update your pipelines to use the v2 versions of the Amazon S3, Microsoft Azure Storage or Google Cloud Storage readers. Refer to the guide Ingesting from Object Storage for details.
Configurable Timeout on Test Pipeline
- Pipeline testing now includes configurable timeouts for both Quick and Full tests. Users can adjust timeout settings to accommodate specific environmental circumstances regarding data arrival. Refer to Test Timeout for details.
Fixes
This release includes numerous bug, stability and security fixes.
Expand to see detail of some important resolved issues here
Stream Processor
- Resolved an issue related to input ordering for nodes that utilize multiple inputs.
Database
- DBM now resumes a data migration process that was interrupted due to a crash, as opposed to losing progress.
SQL2
-
Fixed an issue where an error would be thrown in a SQL2 join query where only the table was referenced, and no
SELECT … FROM [table name]
clause was provided. -
Fixed an issue where a SQL2 query would error if the column in the
WHERE
clause did not appear in theSELECT
clause.
Security
- Various CVEs were remediated as part of this release.
Important Upgrade and Deployment Considerations
Python 3.11 Upgrade
The upgrade to Python 3.11 in the Stream Processor and Data Access Processes brings the following important changes:
-
EmbedPy Removal: Customer code using EmbedPy no longer works.
-
PyKX Upgrade: PyKX has been upgraded from version 2.5.5 to 3.1, which includes breaking changes to PyKX behavior. For more details, refer to this article.
-
Machine Learning API Update:
The
scikit-learn
library has been upgraded from 1.5.2 to 1.6.1. This upgrade introduces the following breaking API changes:adaBoostClassifier
: The SP operator no longer supportsSAMME.R
.randomForestClassifier
: ThemaxFeatures
parameter now defaults tosqrt
instead ofauto
.Time-Based Features
: Time columns are no longer automatically converted to a supported type (integer or float).
getData
The behavior of distinct
when used in the agg
parameter of the arguments of .kxi.getData
is now always returned enlisted
. Refer to the following example:
1) With `agg=enlist`distA`distinct`a` would get the following payload returned:
Old Behavior:
distA
-----
1
0
2
4
New Behavior:
distA
-------
1 0 2 4
2) With `agg=((`distA`distinct`a);(`distB`distinct`b))` both old and new behavior would return this:
distA distB
-----------------
1 0 2 4 4 0 2 1 3
Artifacts
KX Nexus
The KX Downloads Portal is the preferred repository for all use cases, and KX Nexus links will be removed once the KX Nexus repository is decommissioned. Nexus links on this page are provided as a temporary alternative to the KX Downloads Portal for existing users.
Stream Processor
service | Nexus Docker | Downloads Portal Docker |
---|---|---|
Controller | registry.dl.kx.com/kxi-sp-controller:1.14.0 | portal.dl.kx.com/kxi-sp-controller:1.14.0 |
Worker | registry.dl.kx.com/kxi-sp-worker:1.14.0 | portal.dl.kx.com/kxi-sp-worker:1.14.0 |
Coordinator | registry.dl.kx.com/kxi-sp-coordinator:1.14.0 | portal.dl.kx.com/kxi-sp-coordinator:1.14.0 |
Worker-Python | registry.dl.kx.com/kxi-sp-python:1.14.0 | portal.dl.kx.com/kxi-sp-python:1.14.0 |
Worker-ML | registry.dl.kx.com/kxi-ml:1.14.0 | portal.dl.kx.com/kxi-ml:1.14.0 |
Database
service | Nexus Docker | Downloads Portal Docker | Helm Chart |
---|---|---|---|
Data Access Process (DA) | registry.dl.kx.com/kxi-da:1.14.0 | portal.dl.kx.com/kxi-da:1.14.0 | |
Single container | registry.dl.kx.com/kxi-da-single:1.14.0 | portal.dl.kx.com/kxi-da-single:1.14.0 | |
Storage Manager | registry.dl.kx.com/kxi-sm-single:1.14.0 | portal.dl.kx.com/kxi-sm-single:1.14.0 | |
Gateway | registry.dl.kx.com/kxi-sg-gw:1.14.0 | portal.dl.kx.com/kxi-sg-gw:1.14.0 | |
Resource Coordinator | registry.dl.kx.com/kxi-sg-rc:1.14.0 | portal.dl.kx.com/kxi-sg-rc:1.14.0 | |
Aggregator | registry.dl.kx.com/kxi-sg-agg:1.14.0 | portal.dl.kx.com/kxi-sg-agg:1.14.0 | |
Rollback | registry.dl.kx.com/kxi-db-rollback:1.14.0 | portal.dl.kx.com/kxi-db-rollback:1.14.0 | |
Reference Deployments | N/A | Docker Bundle | N/A |
Reference Deployments | N/A | N/A | Kubernetes Bundle |
Reliable Transport
service | Nexus Docker | Downloads Portal Docker | Helm Chart |
---|---|---|---|
Reliable Transport | registry.dl.kx.com/kxi-rt:1.14.0 | portal.dl.kx.com/kxi-rt:1.14.0 | kxi-rt-1.14.0.tgz |
Miscellaneous
service | Nexus Docker | package |
---|---|---|
Metrics Sidecar | registry.dl.kx.com/kxi-sidecar:1.13.0 | |
OpenAPI SDK Generator | registry.dl.kx.com/kxi-openapi-codegen:0.8.0 | https://nexus.dl.kx.com/repository/kx-insights-packages/kxi-openapi-codegen/0.8.0/kxi-openapi-codegen-0.8.0-shaded.jar |