Skip to content

Release Notes - kdb Insights Enterprise 1.1.2

kdb Insights Enterprise 1.1.2 is a patch release that corrects a small number of observability and query issues.

Release Date

2022-07-15

Observability

[FIX] Metrics responses are all now terminated with a newline \n. Absence of the newline caused import issues with some metrics scrapers, for example Azure Monitor.

Explore and Queries

[FIX] The getData API endpoint no longer returns an error when an outputTZ is provided.

[FIX] Queries against the interval tier (idb) around the end-of-day rollover process could return zero results.

Infrastructure

[NEW] The Terraform scripts will now pass the correct platform to the Docker container when running them on Macs

[FIX] The Helm version in the Terraform scripts' Dockerfile has been pinned to a version without AWS compatability issues.

Artifacts

type location
Infrastructure kxi-terraform-1.1.1.tgz
Platform insights-1.1.2.tgz
Operator kxi-operator-1.1.0.tgz
CLI kxicli-0.9.0-py3-none-any.whl
ODBC Driver kodbc 1.1.0
Java SDK java-sdk 1.1.0

Upgrade notes

Data loss of UI assembly definitions for versions earlier than 1.1.1

Assembly definitions built from the Insights UI in versions earlier than 1.1.1 will be lost as part of an uninstall of the application. The assembly components (assemblies, databases, pipelines, reports, schemas, streams) are stored on a PVC in the application. This storage is incorrectly torn down as part of an uninstall, resulting in data loss on re-install or upgrade.

To prevent this data loss the following commands can be run to backup the data. This should be run prior to the uninstall, with kubectl configured to the target cluster and namespace.

kubectl exec insights-kxi-controller-0 -- tar -C /kxic/data -czf - . > kxi-controller-data.tgz

To restore, the user should run the following command post-install.

cat kxi-controller-data.tgz |  kubectl exec insights-kxi-controller-0 --stdin -- tar -C /kxic/data -xzf -

KXI Operator Version

As the version of the KXI Operator has not changed from 1.1.0 to 1.1.2, the command for deploying insights is slightly different than in earlier releases. The older KXI Operator version must be explicitly passed to the CLI.

kxi install run --filepath values.yaml --version 1.1.2 --operator-version 1.1.0

Please see the guide to Installing Insights or Upgrading Insights for more detail.

New functionality and roles

Insights Platform 1.1.1 includes new features permissioned by additional roles. If you are upgrading to 1.1.2 from a version earlier than 1.1.1, please see the section about authentication upgrades for more information about how to incorporate realm changes, including new roles, during an upgrade.

Known Issues

  • If you request too much data in a single getData(), the request will fail. To overcome this issue you should request less data per getData() call. The error message returned will look like the example below
   {"header":{"http":"json","corr":"6dd8f0c5-1895-49c9-a87a-636a834af370","logCorr":"6dd8f0c5-1895-49c9-a87a-636a834af370","client":":10.0.10.76:5050","api":".kxi.getData","protocol":"gw","numRP":1,"ogRcID":"10.0.10.100:5060","to":"2022-04-07T11:41:35.717000000","retryCount":0,"rc":42,"ac":10,"ai":"Agg died"},"payload":[]}
  • The user-defined variables & functions in the Explore tab are shared across users. (We recommend that you prefix all your variables with characters that are unique to your user.)

  • Assemblies applied directly through kubectl cannot be edited in the User Interface. However, they can be queried via the Explore tab.

  • Currently users will be logged out but will not be notified via the UI. If users consistently encounter errors, we suggest they log out and in again.

  • The Explore tab allows users to execute arbitrary code within the cluster. To lock down the environment, this can be restricted with role based authentication using Keycloak. Restrict the roles relating to insights.scratch.*

  • The Explore tab will not print results to the console for multi-line python code. Results are only printed if there is just one line of python code.

  • The drop-down list of assemblies in the first Explore tab opened after applying the first Insights assembly will take a short while to populate. Try again after a few seconds if nothing appears immediately, or close and re-open the Explore tab to see the list and select an assembly.

  • Teardown cleanups of Data (assemblies) via the UI will not fully remove PVCs if there are active Explore tabs targetting that assembly. Close these tabs or remove their underlying sandboxes via the REST API to complete the cleanup.

  • On startup of pods, the following error might be observed once roughly after three minutes of a pod starting up no acct for 3x period, exiting. This stems from a temporary startup job not shutting down correctly. It's independent from the main processes and doesn't indicate any application fault.

  • On initial startup of kdb Insights Enterprise, there may be some noise printed in the logs while the system initialises unable to flush accounting logs. This relates to the capturing of consumption-based license logs and is thrown while all pods get into a running state. It does not indicate any fault in the application and all data should be flushed correctly after a short period.

  • The KX ODBC driver is not currently available for use on Windows, therefore as the Java SDK requires the ODBC driver to be installed, neither SDKs are currently available on Windows.

  • When either the Java SDK or ODBC driver is being used, if Insights Information Service fails to respond to the SDK for at least 1 minute it will deem their own configuration too old and will disconnect and stop publisher into Insights. This can be observed when the control plane is inaccessible (for example during the default GCP maintenance window, 5am UTC, if it hasn’t been disabled). Any application using the SDKs will need to account for this and introduce some restart logic which will trigger the SDK to restart, and force a reconnect attempt by the SDK.

Sample extract from the application log:

2022-05-24 20:30:48.103035      INFO    1       Started replicator to :34.142.40.180:5000, pid 41697
2022-05-24 20:31:48.127821      INFO    1       Process 41697 terminated
2022-05-24 20:31:48.128196      INFO    1       Started replicator to :34.142.40.180:5000, pid 41714
2022-05-24 20:36:12.700002      INFO    1       Process 41714 terminated
2022-05-24 20:36:12.700467      INFO    1       Started replicator to :34.142.40.180:5000, pid 41772
2022-05-25 05:04:50.077286      INFO    0       New configuration fetched
2022-05-25 05:04:50.092259      WARN    0       Config parse error at offset 0: Expected opening bracket
2022-05-25 05:05:50.046686      ERROR   0       Config is too old
2022-05-25 05:05:50.046813      INFO    0       Waiting for the driver to sync the data

Backward Compatibility

Please see the release notes for kdb Insights Enterprise 1.1.0 if you're upgrading from an earlier version, for notes about