Skip to content

Release Notes - kdb Insights Microservices 1.4.0

kdb Insights Microservices 1.4.0 release.

Release Date

2023-03-03

New Features

Reliable Transport

[NEW] A Reliable Transport Microservice is now available. The kdb Insights Reliable Transport (RT) is a microarchitecture for ensuring the reliable streaming of messages. It is designed to satisfy both high availability and high performance requirements.

Stream Processor

[NEW] Ingest large batches of data directly into the historical database. - Use fewer resources. Memory and disk resources greatly reduced during ingest. - New functionality available in the q and Python APIs.

Known Issues

  • If end-of-day processing is interrupted and remains incomplete until the next EOD period, this prevents further EODs from completing and requires manual recovery.

  • Adding a sort attribute to memory or disk is currently not supported. Changing the attribute to be sorted will not apply the attribute and will error for data on disk.

  • Schema changes will fail when attempting to remove all symbol columns from a schema that previously had any.

  • If the Resource Coordinator (RC) times out a request on a dequeue, the displayed message "Unknown reason" should be interpreted as "Timed out before we were able to dequeue".

  • If two processes have the same host and port, then the second one to register will overwrite the first, meaning the first will be invisible to the RC, and consequently never receive any requests.

  • If a reader node is added to a pipeline that invokes .qsp.read.fromFile using chunking, then compilation fails.

Upgrade considerations

Discovery Service

Spring Boot has been updated to version 3.0.0, this upgrade changes how profiles are activated within the application.yaml. If using a spring.profiles.active it should be updated to spring.config.active.on-profile. A working configuration is documented within the Service Discovery Microservice documentation

Full details on the Spring profiles change here

Artifacts

Highlights denote artifacts which have been updated as part of this patch release. All other artifacts are unchanged from the previous major or minor release.

Service Discovery

microservice Docker Kubernetes
Discovery Registry registry.dl.kx.com/kxi-eureka-discovery:1.4.0 https://nexus.dl.kx.com/repository/kx-insights-charts/kxi-discovery-service-1.4.0.tgz
Discovery Proxy registry.dl.kx.com/kxi-discovery-proxy:1.4.0

Stream Processor

microservice Docker qpk
Controller registry.dl.kx.com/kxi-sp-controller:1.4.0
Worker registry.dl.kx.com/kxi-sp-worker:1.4.0 https://nexus.dl.kx.com/repository/kx-insights-packages/kxi-sp-worker/1.2.0/kxi-sp-worker.1.4.0.qpk
Coordinator registry.dl.kx.com/kxi-sp-coordinator:1.4.0
Worker-Python registry.dl.kx.com/kxi-sp-python:1.4.0
Worker-ML registry.dl.kx.com/kxi-ml:1.4.0

Data Access

microservice Docker
Data Access Process (DA) registry.dl.kx.com/kxi-da:1.4.0
Single container registry.dl.kx.com/kxi-da-single:1.4.0

Storage Manager

microservice Docker
Storage Manager registry.dl.kx.com/kxi-sm:1.4.0
End of Interval (EOI) registry.dl.kx.com/kxi-sm-eoi:1.4.0
End of Day (EOD) registry.dl.kx.com/kxi-sm-eod:1.4.0
Database Migration (DBM) registry.dl.kx.com/kxi-sm-dbm:1.4.0
Single Container registry.dl.kx.com/kxi-sm-single:1.4.0

Service Gateway

microservice Docker
Gateway registry.dl.kx.com/kxi-sg-gw:1.4.0
Resource Coordinator registry.dl.kx.com/kxi-sg-rc:1.4.0
Aggregator registry.dl.kx.com/kxi-sg-agg:1.4.0

User interface

microservice Docker Helm Chart
Dashboards registry.dl.kx.com/kxi-gui-dash:1.4.0 https://nexus.dl.kx.com/repository/kx-insights-charts/gui-dash-1.4.0.tgz

Machine Learning

microservice package
ML Registry https://nexus.dl.kx.com/repository/kx-insights-packages/kxi-ml-registry/0.8.0/kxi-ml-registry.0.8.0.qpk
ML Analytics https://nexus.dl.kx.com/repository/kx-insights-packages/kxi-ml-analytics/0.8.0/kxi-ml-analytics.0.8.0.qpk

Reliable Transport

microservice Docker Kubernetes
Reliable Transport registry.dl.kx.com/kxi-rt:1.4.0 https://nexus.dl.kx.com/repository/kx-insights-charts/kxi-rt-1.4.0.tgz

Miscellaneous

microservice Docker package
Metrics Sidecar registry.dl.kx.com/kxi-sidecar:1.4.0-rc1
OpenAPI SDK Generator registry.dl.kx.com/kxi-openapi-codegen:0.8.0 https://nexus.dl.kx.com/repository/kx-insights-packages/kxi-openapi-codegen/0.8.0/kxi-openapi-codegen-0.8.0-shaded.jar