Skip to content

Release Notes - kdb Insights Enterprise 1.1.1

kdb Insights Enterprise 1.1.1 is a patch release that improves usability and stability.

Release Date

2022-06-28

Data (Assemblies)

[FIX] Clearer tooltips for Data status to ensure users can clearly identify when their assemblies are fully deployed.

[FIX] Additional validation has been added to stop re-deployment of Data (assemblies) after schema changes – schema changes on deployed assemblies are not supported in this version of Insights.

[FIX] Additional validation has been added for schemas to ensure they can be deployed successfully.

[FIX] The serviceVolume setting is now mapped correctly in Database and Stream

[FIX] A condition where Stream log archiving could fail has been resolved, ensuring that stream log files do not grow beyond the defined limits.

[FIX] Navigation tree issues where schemas could not be opened and deletion failures were not correctly reflected are both resolved.

Pipelines

[FIX] Nodes requiring passwords no longer store them in the pipeline configuration, they must be entered during pipeline deployment.

[FIX] Mandatory fields in nodes are now enforced in pipelines before deployment.

[FIX] Pipeline re-deployments can now optionally clear state (destroying previously persisted pipeline data, but not data already in the database) to allow edits to be made and the pipeline to be restarted from scratch.

[FIX] The error Cannot read properties of undefined (reading '0') which could be thrown when opening multiple pipeline tabs has been resolved.

[FIX] An issue where long code lines in Map nodes could cause browsers to hang has been resolved.

Explore and Queries

[FIX] Session errors Worker has been closed and r.map are no longer thrown.

[FIX] The error No aggregator available when changing assemblies in an Explore session is no longer thrown.

[FIX] Reference data joins which do not include the foreign column or where the foreign column has a different name are now possible.

[FIX] An issue where certain queries’ WHERE clause would work via REST but fail via Explore has been resolved.

[FIX] An issue where certain queries using count could throw the error Error: noupdate: `.Q `pn has been resolved.

Artifacts

type location
Infrastructure kxi-terraform-1.1.0.tgz
Platform insights-1.1.1.tgz
Operator kxi-operator-1.1.0.tgz
CLI kxicli-0.9.0-py3-none-any.whl
ODBC Driver kodbc 1.1.0
Java SDK java-sdk 1.1.0

Upgrade notes

Data loss of UI assembly definitions

Assembly definitions built from the Insights UI will be lost as part of an uninstall of the application. The assembly components (assemblies, databases, pipelines, reports, schemas, streams) are stored on a PVC in the application. This storage is incorrectly torn down as part of an uninstall, resulting in data loss on re-install or upgrade.

To prevent this data loss the following commands can be run to backup the data. This should be run prior to the uninstall, with kubectl configured to the target cluster and namespace.

kubectl exec insights-kxi-controller-0 -- tar -C /kxic/data -czf - . > kxi-controller-data.tgz

To restore, the user should run the following command post-install.

cat kxi-controller-data.tgz |  kubectl exec insights-kxi-controller-0 --stdin -- tar -C /kxic/data -xzf -

KXI Operator Version

As the version of the KXI Operator has not changed from 1.1.0 to 1.1.1, the command for deploying insights is slightly different than in earlier releases. The older KXI Operator version must be explicitly passed to the CLI.

kxi install run --filepath values.yaml --version 1.1.1 --operator-version 1.1.0

Please see the guide to Installing Insights or Upgrading Insights for more detail.

New functionality and roles

Insights Platform 1.1.1 includes new features permissioned by additional roles. Please see the section about authentication upgrades for more information about how to incorporate realm changes, including new roles, during an upgrade.

Known Issues

  • If you request too much data in a single getData(), the request will fail. To overcome this issue you should request less data per getData() call. The error message returned will look like the example below
   {"header":{"http":"json","corr":"6dd8f0c5-1895-49c9-a87a-636a834af370","logCorr":"6dd8f0c5-1895-49c9-a87a-636a834af370","client":":10.0.10.76:5050","api":".kxi.getData","protocol":"gw","numRP":1,"ogRcID":"10.0.10.100:5060","to":"2022-04-07T11:41:35.717000000","retryCount":0,"rc":42,"ac":10,"ai":"Agg died"},"payload":[]}
  • The user-defined variables & functions in the Explore tab are shared across users. (We recommend that you prefix all your variables with characters that are unique to your user.)

  • Assemblies applied directly through kubectl cannot be edited in the User Interface. However, they can be queried via the Explore tab.

  • Currently users will be logged out but will not be notified via the UI. If users consistently encounter errors, we suggest they log out and in again.

  • The Explore tab allows users to execute arbitrary code within the cluster. To lock down the environment, this can be restricted with role based authentication using Keycloak. Restrict the roles relating to insights.scratch.*

  • The Explore tab will not print results to the console for multi-line python code. Results are only printed if there is just one line of python code.

  • The drop-down list of assemblies in the first Explore tab opened after applying the first Insights assembly will take a short while to populate. Try again after a few seconds if nothing appears immediately, or close and re-open the Explore tab to see the list and select an assembly.

  • On startup of pods, the following error might be observed once roughly after three minutes of a pod starting up no acct for 3x period, exiting. This stems from a temporary startup job not shutting down correctly. It's independent from the main processes and doesn't indicate any application fault.

  • On initial startup of kdb Insights Enterprise, there may be some noise printed in the logs while the system initialises unable to flush accounting logs. This relates to the capturing of consumption-based license logs and is thrown while all pods get into a running state. It does not indicate any fault in the application and all data should be flushed correctly after a short period.

  • The KX ODBC driver is not currently available for use on Windows, therefore as the Java SDK requires the ODBC driver to be installed, neither SDKs are currently available on Windows.

  • When either the Java SDK or ODBC driver is being used, if Insights Information Service fails to respond to the SDK for at least 1 minute it will deem their own configuration too old and will disconnect and stop publisher into Insights. This can be observed when the control plane is inaccessible (for example during the default GCP maintenance window, 5am UTC, if it hasn’t been disabled). Any application using the SDKs will need to account for this and introduce some restart logic which will trigger the SDK to restart, and force a reconnect attempt by the SDK.

Sample extract from the application log:

2022-05-24 20:30:48.103035      INFO    1       Started replicator to :34.142.40.180:5000, pid 41697
2022-05-24 20:31:48.127821      INFO    1       Process 41697 terminated
2022-05-24 20:31:48.128196      INFO    1       Started replicator to :34.142.40.180:5000, pid 41714
2022-05-24 20:36:12.700002      INFO    1       Process 41714 terminated
2022-05-24 20:36:12.700467      INFO    1       Started replicator to :34.142.40.180:5000, pid 41772
2022-05-25 05:04:50.077286      INFO    0       New configuration fetched
2022-05-25 05:04:50.092259      WARN    0       Config parse error at offset 0: Expected opening bracket
2022-05-25 05:05:50.046686      ERROR   0       Config is too old
2022-05-25 05:05:50.046813      INFO    0       Waiting for the driver to sync the data

Backward Compatibility

Please see the release notes for kdb Insights Enterprise 1.1.0 if you're upgrading from an earlier version, for notes about