Skip to content

Upgrading kdb Insights Enterprise

This section details the steps to upgrade the kdb Insights Enterprise from a current install.

If you are upgrading kdb Insights Enterprise on Azure Marketplace please refer to the Azure upgrade guide.

Prerequisites

Warning

The upgrade process requires downtime to ensure data integrity can be guaranteed. This means that before any upgrade is scheduled, a number of pre-flight checks are necessary.

  1. Stop ingestion feeds

    Stop all feeds into the application to avoid data loss with inflight data.

  2. Ensure latest version of kxi-cli is installed

  3. If you are upgrading from 1.1.X to 1.2.0+, ensure you have performed the Keycloak 18 migration steps

  • Pre-Upgrade Versions The currently running versions of kdb Insights Enterprise and Operator should be noted for a rollback scenario

    1. Operator

      helm ls -n kxi-operator

      NAME        NAMESPACE       REVISION    UPDATED                                 STATUS      CHART                       APP VERSION
insights    kxi-operator    1           2022-04-01 15:39:44.955236379 +0000 UTC deployed    kxi-operator-1.0.0          1.0.0

    2. Application 

      helm ls -n $INSIGHTS_NAMESPACE


      NAME        NAMESPACE   REVISION    UPDATED                                 STATUS      CHART                   APP VERSION
insights    kxi         1           2022-04-01 16:57:00.144126754 +0000 UTC deployed    insights-1.0.0          1.0.0

  • Operator version The CLI installs the latest kdb Insights Operator minor version that matches the specified $INSIGHTS_VERSION. You can override this by passing the --operator-version command line option.

  • Install values Retrieve the INSTALL_CONFIG_FILE from the existing with the following command. Update the resulting file to customise the install configuration upon upgrade, including version-specific upgrade considerations. 

    helm get values insights -n $INSIGHTS_NAMESPACE | tail -n +2 > $INSTALL_CONFIG_FILE

  • Version-specific considerations Please see release notes for any specific upgrade considerations for your version of kdb Insights Enterprise.

Variables

Have the following information to hand:

variable example further details
INSIGHTS_NAMESPACE kxi Namespace of existing deployment
INSIGHTS_VERSION 1.1.1 Version to upgrade to
INSTALL_CONFIG_FILE values.yaml Install configuration
PRE_UPGRADE_INSIGHTS_VERSION 1.0.0 Current application version
PRE_UPGRADE_OPERATOR_VERSION 1.0.0 Current Operator version

Upgrading

The CLI is used to perform upgrades. This process does not remove the persisted data backing kdb Insights Enterprise configuration or databases. These remain as persisted volumes within the Kubernetes cluster.

The steps performed by the CLI are:

  • Backup assemblies
  • Teardown assemblies
  • Upgrade application, Operator and CRDs
  • Restore assemblies

If there are no assemblies running, the CLI upgrades the base system and skips the assembly steps.

To initiate the upgrade, run the following command. This steps through the upgrade, prompting you as required.

kxi install upgrade --filepath $INSTALL_CONFIG_FILE --version $INSIGHTS_VERSION

Backing up assemblies
Persisted assembly definitions for ['dfx-assembly', 'iot-assembly'] to <ASSEMBLY STATE FILE>
...
Upgrading insights and operator
...
Reapplying assemblies
...
Upgrade to version 1.1.1 complete

Post-upgrade checks

Check the system is running correctly.

  1. Expected version of kdb Insights Enterprise is running 

    kubectl describe cm insights-config | grep version
version: insights-1.1.1

  2. Operator installed 

    helm ls -n kxi-operator


    NAME        NAMESPACE       REVISION    UPDATED                                 STATUS      CHART                       APP VERSION
insights    kxi-operator    1           2022-04-11 15:39:44.955236379 +0000 UTC deployed    kxi-operator-1.1.0          1.1.0

  3. Application installed 

    helm ls -n $INSIGHTS_NAMESPACE


    NAME        NAMESPACE   REVISION    UPDATED                                 STATUS      CHART                   APP VERSION
insights    kxi         1           2022-04-11 16:57:00.144126754 +0000 UTC deployed    insights-1.1.1          1.1.1

  4. Operator in running state 

    kubectl get pods -n kxi-operator


    NAME                                     READY   STATUS    RESTARTS   AGE
insights-kxi-operator-6df4bcfddc-gnnjj   1/1     Running   0          85m
insights-kxi-operator-6df4bcfddc-lqmrl   1/1     Running   0          85m
insights-kxi-operator-6df4bcfddc-z45wq   1/1     Running   0          85m

  5. Application in running state 

    kubectl get pods -n $INSIGHTS_NAMESPACE


    NAME                                            READY   STATUS      RESTARTS   AGE
insights-aggregator-9959bdc87-4pg7z             2/2     Running     0          9m47s
...
...
insights-sg-gateway-596b486d9f-pmhf4            1/1     Running     0          9m47s
insights-sg-gateway-596b486d9f-qvzbr            1/1     Running     0          9m47s

  6. Assembly CRDs redeployed 

    kubectl get crds | grep insights.kx.com


    assemblies.insights.kx.com                   2022-04-11T15:23:53Z
assemblyresources.insights.kx.com            2022-04-11T15:23:54Z

  7. Ensure assemblies are up and running successfully. The CLI restores assemblies that were running before the upgrade was initiated. The example below shows two sample assemblies; you may have different assemblies, or none at all.

    kubectl get asm

    NAME           DESCRIPTION      READY   STATUS   AGE
dfx-assembly   A KXI Assembly   True             28m
iot-assembly   A KXI Assembly   True             28m

Rollback

A previous installation state can be recoverd by tearing down and reinstalling insights. This process will not remove the persisted data backing kdb Insights Enterprise configuration or databases. These remain as persisted volumes within the Kubernetes cluster.

  1. Backup any assemblies that may be running 
    $ kxi assembly backup
Persisted assembly definitions for ['dfx-assembly', 'iot-assembly'] to <ASSEMBLY STATE FILE>
  2. Teardown assemblies named in the output of the backup command 
    $ kxi assembly teardown --name dfx-assembly
Tearing down assembly dfx-assembly
Are you sure you want to teardown dfx-assembly [y/N]: y
$ kxi assembly teardown --name iot-assembly
Tearing down assembly iot-assembly
Are you sure you want to teardown iot-assembly [y/N]: y
  3. Teardown current insights 
    $ kxi install delete --uninstall-operator
  4. Install insights with the desired version and values file 
    $ kxi install run --filepath $INSTALL_CONFIG_FILE --version $PRE_UPGRADE_INSIGHTS_VERSION --operator-version $PRE_UPGRADE_OPERATOR_VERSION
  5. Restore assemblies 
    $ kxi assembly deploy --filepath <ASSEMBLY STATE FILE> --use-kubeconfig

Note that an initUser in the INSTALL_CONFIG_FILE will be re-imported and the password reset to that in the file.

Upgrading to an older version is not directly supported. This will be blocked in future releases and replaced by a supported rollback command.

Failed upgrades

Assembly state files are retained upon upgrade failure. The following is an example output from a failed upgrade. 

...
Persisted assembly definitions for ['dfx-assembly', 'iot-assembly'] to /tmp/tmp7ny84esa-kxi-assembly-state.yaml
...
Error: Command '['helm', 'upgrade', '--install', '-f', './values.yaml', 'insights', 'kx-insights/insights', '--version', '1.1.1', '--namespace', 'insights']' returned non-zero exit status 1.

After recovering the state of kdb Insights Enterprise, restore assemblies to their running state by executing the following command, replacing the assembly state file name with that created by the failed upgrade. 

kxi assembly create --filepath /tmp/tmp7ny84esa-kxi-assembly-state.yaml