Release Notes - kdb Insights Enterprise 1.3.2
Release for kdb Insights Enterprise.
[Beta] Compressed files (GZIP) can now be decoded using the q and Python APIs.
Multiple file paths can be added to a cloud storage Reader node using the
+button in the UI, or via the q and Python APIs.
Wildcard support using
*is now available when using a single file path with the Amazon S3 Reader.
CSV encoding is now available, using a new node in the UI, or via the q and Python APIs.
It is now possible to write to Amazon S3 buckets, using a new node in the UI, or via the q and Python APIs.
Pipeline lifecycle events can be subscribed to via the q API. This can be used, for example, to invoke a callback once files have been processed by a pipeline.
- Queries will no longer time out after 60 seconds at the gateway. Longer timeouts can be configured in
getDataAPI calls using the
- An issue where DAPs could enter low-memory mode earlier than they should has been resolved.
|ODBC Driver||kodbc 1.3.0|
|Java SDK||java-sdk 1.3.0|
When upgrading kdb Insights Enterprise, assemblies deployed from the UI in earlier versions will be restarted but will not be automatically upgraded. To upgrade these assemblies, please restart them from the UI post-upgrade.
Keycloak Config CLI
It is recommended to enable the Keycloak Config CLI when upgrading to ensure that any realm changes are imported.
This can be enabled by setting:
keycloak: keycloakConfigCli: enabled: true
in your values file if you are deploying Keycloak as a part of kdb Insights Enterprise.
If you use a shared Keycloak instance, this can be enabled by setting:
keycloak-config-cli: enabled: true
in your values file.
Keycloak initUser password reset
Enabling the Keycloak Config CLI will cause the initUser's password to be reset. It will be reset to the value of the
Default password policy
As of 1.3.0 a default password policy is now being enforced.
The default policy is:
- At least one uppercase letter
- At least one lowercase letter
- At least one symbol
- At least one number
- Minimum length of 14 characters or greater
Information about how the policy can be configured and adjusted can be found here
If you are upgrading from an earlier version, and want the default password policy to be applied, the keycloak config CLI must be enabled.
initUser password must satisfy the policy if it is enabled
Default Keycloak Credentials
As of 1.3.0 the Keycloak
initClient credentials are no longer defaulted within kdb Insights Enterprise
values.yaml. These defaults would previously create the
demoinsights user and
test-publisher client on a new deployment of kdb Insights Enterprise.
Users who currently set
initClient.enabled=true within their own
values.yaml may receive the following errors at deploy time:
Keycloak initUser has been enabled The following fields are required to be set .Values.keycloak.initUser.name .Values.keycloak.initUser.auth Keycloak initClient has been enabled The following fields are required to be set .Values.keycloak.initClient.clientId .Values.keycloak.initClient.clientSecret
If enabling the
initUser you are required to set:
keycloak: initUser: enabled: true name: "initUsername" auth: "initUserPassword"
initUser.authmust satisfy the policy requirements.
If enabling the
initClient you are required to set:
keycloak: initClient: enabled: true clientId: "initClientID" clientSecret: "initClientSecret"
Internal Network LoadBalancers
As of 1.3.0 by default annotations are added to Service resources of type
These annotations restrict access to the
LoadBalancers from outside the cluster.
To disable these annotations and permit access from outside the cluster, the user is required to set:
global: service: useInternalLBAnnotations: false
For additional configuration options see here
blockSize configuration within an assembly
spec.tables.<table>.blockSize has been updated with the following semantics:
- if unset: all data received in an interval will be buffered in RAM within SM, written down at the end of the interval
- this has the highest performance, but has no RAM limit on received data
- if set: once a table's rows surpass the configured limit, buffered data will be flushed to disk to release RAM
- the smaller this number is set to, the worse ingest performance but stronger RAM limits - this should be balanced
blockSize was ignored, always buffering all data in memory. To reproduce previous behaviour, unset the
blockSize field in the assembly for each table.
On startup of pods, the following error might be observed once roughly after three minutes of a pod starting up
no acct for 3x period, exiting. This stems from a temporary startup job not shutting down correctly. It is independent from the main processes and does not indicate any application fault.
On initial startup of kdb Insights Enterprise, there may be some noise printed in the logs while the system initialises
unable to flush accounting logs. This relates to the capturing of consumption-based license logs and is thrown while all pods get into a running state. It does not indicate any fault in the application and all data should be flushed correctly after a short period.
- Setting SM replicas (defining
sizegreater than 1) in assembly YAML will cause writedown/storage and query problems. The
sizeparameter for SM should always be set to 1.
sm: size: 1
- If the cluster and/or resource configuration for the
kxi-discovery-serviceis limited, a race condition can occur at startup causing the Discovery Service to be in a crash/restart loop. This can be solved by giving the Discovery Service additional CPU and memory resources; full details on setting custom resources can be found here
- Upon upgrade or downgrade, the API Gateway containers may enter a CrashLoopBackOff state. Resources can be reapplied by performing a 'rollback' to the upgraded version. Get the upgraded version by looking at the output from:
'Rollback' (re-apply resources) to the upgraded version:
helm rollback <release name> <current revision>
There are two known issues with the UI logout.
- After performing a logout action, the user will not be redirected back to the login screen.
- In the case a user was logged out due to inactivity, a "Logout failed" error might be observed
In both of these cases, the logout has been successful and the user can re-login by navigating back to the main page, i.e.
Please see the release notes for kdb Insights Enterprise 1.1.0 and kdb Insights Enterprise 1.2.0 if you're upgrading from versions earlier than 1.1.0 or 1.2.0, for notes about