Release Notes - kdb Insights Microservices 1.4.0
kdb Insights Microservices 1.4.0 release.
Release Date
2023-03-03
New Features
Reliable Transport
[NEW] A Reliable Transport Microservice is now available. The kdb Insights Reliable Transport (RT) is a microarchitecture for ensuring the reliable streaming of messages. It is designed to satisfy both high availability and high performance requirements.
Stream Processor
[NEW] Ingest large batches of data directly into the historical database. - Use fewer resources. Memory and disk resources greatly reduced during ingest. - New functionality available in the q and Python APIs.
Known Issues
-
If end-of-day processing is interrupted and remains incomplete until the next EOD period, this prevents further EODs from completing and requires manual recovery.
-
Adding a sort attribute to memory or disk is currently not supported. Changing the attribute to be sorted will not apply the attribute and will error for data on disk.
-
Schema changes will fail when attempting to remove all symbol columns from a schema that previously had any.
-
If the Resource Coordinator (RC) times out a request on a dequeue, the displayed message "Unknown reason" should be interpreted as "Timed out before we were able to dequeue".
-
If two processes have the same host and port, then the second one to register will overwrite the first, meaning the first will be invisible to the RC, and consequently never receive any requests.
- If a reader node is added to a pipeline that invokes
.qsp.read.fromFile
using chunking, then compilation fails.
Upgrade considerations
Discovery Service
Spring Boot has been updated to version 3.0.0, this upgrade changes how profiles are activated within the application.yaml. If using a spring.profiles.active
it should be updated to spring.config.active.on-profile
. A working configuration is documented within the Service Discovery Microservice documentation
Full details on the Spring profiles change here
Artifacts
Highlights denote artifacts which have been updated as part of this patch release. All other artifacts are unchanged from the previous major or minor release.
Service Discovery
microservice | Docker | Kubernetes |
---|---|---|
Discovery Registry | registry.dl.kx.com/kxi-eureka-discovery:1.4.0 | https://nexus.dl.kx.com/repository/kx-insights-charts/kxi-discovery-service-1.4.0.tgz |
Discovery Proxy | registry.dl.kx.com/kxi-discovery-proxy:1.4.0 |
Stream Processor
microservice | Docker | qpk |
---|---|---|
Controller | registry.dl.kx.com/kxi-sp-controller:1.4.0 | |
Worker | registry.dl.kx.com/kxi-sp-worker:1.4.0 | https://nexus.dl.kx.com/repository/kx-insights-packages/kxi-sp-worker/1.2.0/kxi-sp-worker.1.4.0.qpk |
Coordinator | registry.dl.kx.com/kxi-sp-coordinator:1.4.0 | |
Worker-Python | registry.dl.kx.com/kxi-sp-python:1.4.0 | |
Worker-ML | registry.dl.kx.com/kxi-ml:1.4.0 |
Data Access
microservice | Docker |
---|---|
Data Access Process (DA) | registry.dl.kx.com/kxi-da:1.4.0 |
Single container | registry.dl.kx.com/kxi-da-single:1.4.0 |
Storage Manager
microservice | Docker |
---|---|
Storage Manager | registry.dl.kx.com/kxi-sm:1.4.0 |
End of Interval (EOI) | registry.dl.kx.com/kxi-sm-eoi:1.4.0 |
End of Day (EOD) | registry.dl.kx.com/kxi-sm-eod:1.4.0 |
Database Migration (DBM) | registry.dl.kx.com/kxi-sm-dbm:1.4.0 |
Single Container | registry.dl.kx.com/kxi-sm-single:1.4.0 |
Service Gateway
microservice | Docker |
---|---|
Gateway | registry.dl.kx.com/kxi-sg-gw:1.4.0 |
Resource Coordinator | registry.dl.kx.com/kxi-sg-rc:1.4.0 |
Aggregator | registry.dl.kx.com/kxi-sg-agg:1.4.0 |
User interface
microservice | Docker | Helm Chart |
---|---|---|
Dashboards | registry.dl.kx.com/kxi-gui-dash:1.4.0 | https://nexus.dl.kx.com/repository/kx-insights-charts/gui-dash-1.4.0.tgz |
Machine Learning
Reliable Transport
microservice | Docker | Kubernetes |
---|---|---|
Reliable Transport | registry.dl.kx.com/kxi-rt:1.4.0 | https://nexus.dl.kx.com/repository/kx-insights-charts/kxi-rt-1.4.0.tgz |
Miscellaneous
microservice | Docker | package |
---|---|---|
Metrics Sidecar | registry.dl.kx.com/kxi-sidecar:1.4.0-rc1 | |
OpenAPI SDK Generator | registry.dl.kx.com/kxi-openapi-codegen:0.8.0 | https://nexus.dl.kx.com/repository/kx-insights-packages/kxi-openapi-codegen/0.8.0/kxi-openapi-codegen-0.8.0-shaded.jar |