Skip to content

Release Notes - KX Insights Microservices 1.2.0

KX Insights Microservices 1.2.0 release.

Release Date

2022-09-02

Changes

Storage Manager & Data Access

[NEW] Ability for Data Access tiers (RDB, IDB, HDB) to share compute resources by using mountList within Assembly configuration rather than mountName.

Stream Processor

[NEW] MQTT QoS 0 Reader has been added.

Known Issues

  • If you request too much data in a single getData(), the request will fail. To overcome this issue you should request less data per getData() call. The error message returned will look like the example below 
   {"header":{"http":"json","corr":"6dd8f0c5-1895-49c9-a87a-636a834af370","logCorr":"6dd8f0c5-1895-49c9-a87a-636a834af370","client":":10.0.10.76:5050","api":".kxi.getData","protocol":"gw","numRP":1,"ogRcID":"10.0.10.100:5060","to":"2022-04-07T11:41:35.717000000","retryCount":0,"rc":42,"ac":10,"ai":"Agg died"},"payload":[]}
  • .qsp.read.fromDatabase cannot be run in a pipeline to query an Insights database. The following example code can be used as a workaround to query data (ex. reference data to be used within a pipeline).
onResponse:{[hdr;data]
if[0 <> hdr`ac;
    .sp.log.error ("Query failed host=%s, api=%s, code=%n, error=%s";hdr`gw; hdr`api; hdr`ac; hdr`ai)];
publish data
}
.qsp.onStart {
    h:hopen hsym `$.spenv.getReleaseName[],"-sg-gateway:5050";
    res: neg[h] (`.kxi.getData; (enlist `table)!(enlist `instruments); `onResponse; ()!());
}
.qsp.run .qsp.read.fromCallback[`publish] .qsp.write.toConsole[]
  • If metrics reporting is enabled within a microservice, along with the handler for .z.ps, services calling through the .z.ps with 0(f;x) will throw a type error due to a failure in the handler to return correctly. The workaround if metrics reporting is enabled is to disable the .z.ps event handler in the metrics configuration for all microservices.
"metrics": {
    "enabled": "true",
    "frequency": 5,
    "handler": {
        "pc": true,
        "pg": true,
        "ph": true,
        "po": true,
        "pp": true,
        "ps": false,
        "ts": true,
        "wc": true,
        "wo": true,
        "ws": true
    }
}

Upgrade considerations

  • If using a kdb+ tick tickerplant, the schema for database signals has changed to the following, which must be updated in the tick sym.q
(`$"_prtnEnd") set ([] time:"n"$(); sym:`$(); signal:`$(); endTS:"p"$(); opts:())
(`$"_reload") set ([] time:"n"$(); sym:`$(); mount:`$(); params:())

Artifacts

Service Discovery

microservice Docker Helm Chart
Discovery Registry registry.dl.kx.com/kxi-eureka-discovery:1.2.0 https://nexus.dl.kx.com/repository/kx-insights-charts/kxi-discovery-service-1.2.0.tgz
Discovery Proxy registry.dl.kx.com/kxi-discovery-proxy:1.2.0

Stream Processor

microservice Docker qpk
Controller registry.dl.kx.com/kxi-sp-controller:1.2.0
Worker registry.dl.kx.com/kxi-sp-worker:1.2.0 https://nexus.dl.kx.com/repository/kx-insights-packages/kxi-sp-worker/1.2.0/kxi-sp-worker.1.2.0.qpk
Coordinator registry.dl.kx.com/kxi-sp-coordinator:1.2.0
Worker-Python registry.dl.kx.com/kxi-sp-python:1.2.0
Worker-ML registry.dl.kx.com/kxi-ml:1.2.0

Data Access

microservice Docker
Data Access Process (DA) registry.dl.kx.com/kxi-da:1.2.0
Single container registry.dl.kx.com/kxi-da-single:1.2.0

Storage Manager

microservice Docker
Storage Manager registry.dl.kx.com/kxi-sm:1.2.0
End of Interval (EOI) registry.dl.kx.com/kxi-sm-eoi:1.2.0
End of Day (EOD) registry.dl.kx.com/kxi-sm-eod:1.2.0
Database Migration (DBM) registry.dl.kx.com/kxi-sm-dbm:1.2.0
Single Container registry.dl.kx.com/kxi-sm-single:1.2.0

Service Gateway

microservice Docker
Gateway registry.dl.kx.com/kxi-sg-gw:1.2.0
Resource Coordinator registry.dl.kx.com/kxi-sg-rc:1.2.0
Aggregator registry.dl.kx.com/kxi-sg-agg:1.2.0

User interface

microservice Docker Helm Chart
Dashboards registry.dl.kx.com/kxi-gui-dash:1.1.2 https://nexus.dl.kx.com/repository/kx-insights-charts/gui-dash-1.2.0.tgz
Gateway registry.dl.kx.com/kxi-gui-gateway:1.1.2 https://nexus.dl.kx.com/repository/kx-insights-charts/gui-gateway-1.2.0.tgz
Data registry.dl.kx.com/kxi-gui-data:1.1.2 https://nexus.dl.kx.com/repository/kx-insights-charts/gui-data-1.2.0.tgz

Machine Learning

microservice package
ML Registry https://nexus.dl.kx.com/repository/kx-insights-packages/kxi-ml-registry/0.8.0/kxi-ml-registry.0.8.0.qpk
ML Analytics https://nexus.dl.kx.com/repository/kx-insights-packages/kxi-ml-analytics/0.8.0/kxi-ml-analytics.0.8.0.qpk

Miscellaneous

microservice Docker package
Metrics Sidecar registry.dl.kx.com/kxi-sidecar:1.2.0
OpenAPI SDK Generator registry.dl.kx.com/kxi-openapi-codegen:0.8.0 https://nexus.dl.kx.com/repository/kx-insights-packages/kxi-openapi-codegen/0.8.0/kxi-openapi-codegen-0.8.0-shaded.jar