Skip to content

Release Notes - kdb Insights Microservices 1.2.1

kdb Insights Microservices 1.2.1 release.

Release Date

2022-10-20

Storage Manager

[FIX] An upgrade issue affecting migrations from kdb Insights Microservices 1.1 using object storage tiers has been fixed.

[FIX] A problem where underlying system calls could cause Storage Manager to hang during EOI or EOD has been resolved.

[FIX] Discovery issues with DAPs when Storage Manager was used in sm-single mode have been resolved.

[FIX] Storage Manager now retries if it's unable to initially reach the Reliable Transport.

Licensing

[FIX] Airgap licensing problems due to lack of support for regional AWS STS endpoints have been resolved.

Known Issues

  • If you request too much data in a single getData(), the request will fail. To overcome this issue you should request less data per getData() call. The error message returned will look like the example below: 
   {"header":{"http":"json","corr":"6dd8f0c5-1895-49c9-a87a-636a834af370","logCorr":"6dd8f0c5-1895-49c9-a87a-636a834af370","client":":10.0.10.76:5050","api":".kxi.getData","protocol":"gw","numRP":1,"ogRcID":"10.0.10.100:5060","to":"2022-04-07T11:41:35.717000000","retryCount":0,"rc":42,"ac":10,"ai":"Agg died"},"payload":[]}
  • .qsp.read.fromDatabase cannot be run in a pipeline to query a kdb Insights database. The following example code can be used as a workaround to query data (ex. reference data to be used within a pipeline).
onResponse:{[hdr;data]
    if[0 <> hdr`ac;
        .sp.log.error ("Query failed host=%s, api=%s, code=%n, error=%s";hdr`gw; hdr`api; hdr`ac; hdr`ai)];
    publish data
    }
.qsp.onStart {
    h:hopen hsym `$"sg-gateway-host:5050";
    res: neg[h] (`.kxi.getData; (enlist `table)!(enlist `instruments); `onResponse; (0#`)!());
    }
.qsp.run .qsp.read.fromCallback[`publish] .qsp.write.toConsole[]
  • If metrics reporting is enabled within a microservice, along with the handler for .z.ps, services calling through the .z.ps with 0(f;x) will throw a type error due to a failure in the handler to return correctly. The workaround, if metrics reporting is enabled, is to disable the .z.ps event handler in the metrics configuration for all microservices.
"metrics": {
    "enabled": "true",
    "frequency": 5,
    "handler": {
        "pc": true,
        "pg": true,
        "ph": true,
        "po": true,
        "pp": true,
        "ps": false,
        "ts": true,
        "wc": true,
        "wo": true,
        "ws": true
    }
}

Upgrade considerations

  • If using a kdb+ tick tickerplant, the schema for database signals has changed to the following, which must be updated in the tick sym.q
(`$"_prtnEnd") set ([] time:"n"$(); sym:`$(); signal:`$(); endTS:"p"$(); opts:())
(`$"_reload") set ([] time:"n"$(); sym:`$(); mount:`$(); params:(); asm:`$())

Artifacts

Highlights denote artifacts which have been updated as part of this patch release. All other artifacts are unchanged from the previous major or minor release.

Service Discovery

microservice Docker Helm Chart
Discovery Registry registry.dl.kx.com/kxi-eureka-discovery:1.2.1 https://nexus.dl.kx.com/repository/kx-insights-charts/kxi-discovery-service-1.2.1.tgz
Discovery Proxy registry.dl.kx.com/kxi-discovery-proxy:1.2.0

Stream Processor

microservice Docker qpk
Controller registry.dl.kx.com/kxi-sp-controller:1.2.0
Worker registry.dl.kx.com/kxi-sp-worker:1.2.0 https://nexus.dl.kx.com/repository/kx-insights-packages/kxi-sp-worker/1.2.0/kxi-sp-worker.1.2.0.qpk
Coordinator registry.dl.kx.com/kxi-sp-coordinator:1.2.0
Worker-Python registry.dl.kx.com/kxi-sp-python:1.2.0
Worker-ML registry.dl.kx.com/kxi-ml:1.2.0

Data Access

microservice Docker
Data Access Process (DA) registry.dl.kx.com/kxi-da:1.2.1
Single container registry.dl.kx.com/kxi-da-single:1.2.1

Storage Manager

microservice Docker
Storage Manager registry.dl.kx.com/kxi-sm:1.2.1
End of Interval (EOI) registry.dl.kx.com/kxi-sm-eoi:1.2.1
End of Day (EOD) registry.dl.kx.com/kxi-sm-eod:1.2.1
Database Migration (DBM) registry.dl.kx.com/kxi-sm-dbm:1.2.1
Single Container registry.dl.kx.com/kxi-sm-single:1.2.1

Service Gateway

microservice Docker
Gateway registry.dl.kx.com/kxi-sg-gw:1.2.1
Resource Coordinator registry.dl.kx.com/kxi-sg-rc:1.2.1
Aggregator registry.dl.kx.com/kxi-sg-agg:1.2.1

User interface

microservice Docker Helm Chart
Dashboards registry.dl.kx.com/kxi-gui-dash:1.1.2 https://nexus.dl.kx.com/repository/kx-insights-charts/gui-dash-1.2.0.tgz
Gateway registry.dl.kx.com/kxi-gui-gateway:1.1.2 https://nexus.dl.kx.com/repository/kx-insights-charts/gui-gateway-1.2.0.tgz
Data registry.dl.kx.com/kxi-gui-data:1.1.2 https://nexus.dl.kx.com/repository/kx-insights-charts/gui-data-1.2.0.tgz

Machine Learning

microservice package
ML Registry https://nexus.dl.kx.com/repository/kx-insights-packages/kxi-ml-registry/0.8.0/kxi-ml-registry.0.8.0.qpk
ML Analytics https://nexus.dl.kx.com/repository/kx-insights-packages/kxi-ml-analytics/0.8.0/kxi-ml-analytics.0.8.0.qpk

Miscellaneous

microservice Docker package
Metrics Sidecar registry.dl.kx.com/kxi-sidecar:1.2.0
OpenAPI SDK Generator registry.dl.kx.com/kxi-openapi-codegen:0.8.0 https://nexus.dl.kx.com/repository/kx-insights-packages/kxi-openapi-codegen/0.8.0/kxi-openapi-codegen-0.8.0-shaded.jar