Upgrading kdb Insights Enterprise
This section details the steps to upgrade kdb Insights Enterprise from a current install.
If you are upgrading kdb Insights Enterprise on Azure Marketplace please refer to the guide here.
Preflight
Warning
Currently the upgrade process will require downtime to ensure data integrity can be guaranteed. This means that before any upgrade is scheduled, a number of pre-flight checks are necessary
-
Stop ingestion feeds
Stop all feeds into kdb Insights Enterprise, to avoid data loss with in-flight data.
-
Ensure latest version of kxi-cli is installed
-
Ensure you have performed the Keycloak 18 migration steps if you are upgrading from 1.1.X to 1.2.0+
Variables
Having the following information to hand will help smooth the process
variable | example | further details |
---|---|---|
INSIGHTS_NAMESPACE | kxi | Namespace kdb Insights Enterprise deployed to |
INSIGHTS_VERSION | 1.1.1 | Version of kdb Insights Enterprise to upgrade to |
INSTALL_CONFIG_FILE | values.yaml | Install configuration created on initial install |
PRE_UPGRADE_INSIGHTS_VERSION | 1.0.0 | Current kdb Insights Enterprise version |
PRE_UPGRADE_OPERATOR_VERSION | 1.0.0 | Current kxi-operator version |
Pre-Upgrade Versions
The currently running versions of KXI Operator and KXI base charts should be noted for a rollback scenario
-
KXI Operator
helm ls -n kxi-operator
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION insights kxi-operator 1 2022-04-01 15:39:44.955236379 +0000 UTC deployed kxi-operator-1.0.0 1.0.0
-
KXI base charts
helm ls -n $INSIGHTS_NAMESPACE
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION insights kxi 1 2022-04-01 16:57:00.144126754 +0000 UTC deployed insights-1.0.0 1.0.0
Operator Version
The CLI will install the latest operator minor version that matches the specified $INSIGHTS_VERSION
This can be overridden by passing the --operator-version
command line option.
Authentication Upgrades
By default the Keycloak realm configuration is not automatically re-imported on upgrade. This is to avoid resetting runtime changes made to the default user account back to realm defaults but also means any additional roles added to kdb Insights Enterprise with the upgrade will not be imported. It is recommended you enable this realm import when upgrading; and if you are actively using the the default user, review and update these manually within the Keycloak UI.
To enable keycloak realm updates, add the following to your $INSTALL_CONFIG_FILE
keycloak:
keycloakConfigCli:
enabled: true
Please see the section about authentication upgrades for more information about how to incorporate realm changes, during an upgrade.
Version-Specific Considerations
Please see Release Notes for any specific upgrade considerations for your version of kdb Insights Enterprise.
Upgrading
The kxi-cli is used to perform upgrades. The steps to upgrade kdb Insights Enterprise involves removing the current deployment and reinstalling the new version. This process does not remove the persisted data backing kdb Insights Enterprise configuration or databases. These remain as Persisted Volumes within the Kubernetes cluster.
The steps performed by the kxi cli are:
- Backup assemblies
- Teardown assemblies
- Upgrade kdb Insights Enterprise, operator & CRDs
- Restore assemblies
To initiate the upgrade, run the following command. This steps through the upgrade, prompting the user where required
kxi install upgrade --filepath $INSTALL_CONFIG_FILE --version $INSIGHTS_VERSION
Backup And Teardown Assemblies
The state of running assemblies are persisted to a local file. Respond y
to the prompt asking to delete each assembly. This will teardown the running assemblies. Persisted data will be retained upon completion of the upgrade. The upgrade will be exited and assembly state restored if the user responds n
to any of these prompts.
Backing up assemblies
Persisted assembly definitions for ['dfx-assembly', 'iot-assembly'] to kxi-assembly-state.yaml
Tearing down assemblies
Assembly data will be persisted and state will be recovered post-upgrade
Tearing down assembly dfx-assembly
Are you sure you want to teardown dfx-assembly [y/N]: y
Waiting for assembly to be torn down [------------------------------------] 0%
Tearing down assembly iot-assembly
Are you sure you want to teardown iot-assembly [y/N]: y
Waiting for assembly to be torn down [------------------------------------] 0%
Upgrading the Operator and kdb Insights Enterprise
Respond y
to the prompt asking to install the kxi-operator. This will also upgrade the CRDs. The CLI will then upgrade kdb Insights Enterprise and restore assemblies to their running state
Upgrading insights and operator
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "kx-insights" chart repository
Update Complete. ⎈Happy Helming!⎈
kxi-operator already installed with version kxi-operator-1.0.0
Do you want to install kxi-operator version 1.1.0? [Y/n]: y
Installing chart kx-insights/kxi-operator version 1.1.0 with values file from values.yaml
Release "insights" has been upgraded. Happy Helming!
NAME: insights
NAMESPACE: kxi-operator
STATUS: deployed
REVISION: 2
TEST SUITE: None
NOTES:
Deploys the KX Insights Operator.
KX Insights already installed with version insights-1.0.0
Installing chart kx-insights/insights version 1.1.1 with values file from values.yaml
Release "insights" has been upgraded. Happy Helming!
NAME: insights
NAMESPACE: kxi
STATUS: deployed
REVISION: 2
NOTES:
This chart deploys KX Insights base components
Reapplying assemblies
Submitting assembly from kxi-assembly-state.yaml
Submitting assembly dfx-assembly
Custom assembly resource dfx-assembly created!
Submitting assembly iot-assembly
Custom assembly resource iot-assembly created!
Upgrade to version 1.1.1 complete
Post-Upgrade Checks
Check the system is running correctly.
-
Expected version of kdb Insights Enterprise is running
kubectl describe cm insights-config | grep version version: insights-1.1.1
-
KXI Operator installed
helm ls -n kxi-operator
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION insights kxi-operator 1 2022-04-11 15:39:44.955236379 +0000 UTC deployed kxi-operator-1.1.0 1.1.0
-
KXI base charts installed
helm ls -n $INSIGHTS_NAMESPACE
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION insights kxi 1 2022-04-11 16:57:00.144126754 +0000 UTC deployed insights-1.1.1 1.1.1
-
KXI Operator in running state
kubectl get pods -n kxi-operator
NAME READY STATUS RESTARTS AGE insights-kxi-operator-6df4bcfddc-gnnjj 1/1 Running 0 85m insights-kxi-operator-6df4bcfddc-lqmrl 1/1 Running 0 85m insights-kxi-operator-6df4bcfddc-z45wq 1/1 Running 0 85m
-
kdb Insights Enterprise in running state
kubectl get pods -n $INSIGHTS_NAMESPACE
NAME READY STATUS RESTARTS AGE insights-aggregator-9959bdc87-4pg7z 2/2 Running 0 9m47s ... ... insights-sg-gateway-596b486d9f-pmhf4 1/1 Running 0 9m47s insights-sg-gateway-596b486d9f-qvzbr 1/1 Running 0 9m47s
-
KXI Assembly CRDs redeployed
kubectl get crds | grep insights.kx.com
assemblies.insights.kx.com 2022-04-11T15:23:53Z assemblyresources.insights.kx.com 2022-04-11T15:23:54Z
-
Ensure assembly is up and running successfully
kubectl get asm
NAME DESCRIPTION READY STATUS AGE dfx-assembly A KXI Assembly True 28m iot-assembly A KXI Assembly True 28m
Rollback
Should the upgrade fail at any stage, kdb Insights Enterprise can be reverted to its previous state by re-running the upgrade command, passing the pre-upgrade versions
kxi install upgrade --filepath $INSTALL_CONFIG_FILE --version $PRE_UPGRADE_INSIGHTS_VERSION --operator-version $PRE_UPGRADE_OPERATOR_VERSION
Assemblies can be restored to their running state by executing
kxi assembly create --filepath kxi-assembly-state.yaml