Roll back an upgrade

This guide details the procedure for rolling back the Kx Platform after an upgrade. Please note that when an upgrade fails at deploy time the rollback is handled automatically by the install script. This guide focuses on the steps required to roll back an upgraded system to a previously installed version.

How the rollback works

The Kx Platform consists of three different types of content:

Data

The data generated by the system.

The rollback procedure does not cover rolling back data generated by the system since the upgrade.

This is solution-specific and outside of the scope of this document.

Environment config

Applications and binaries are configured via Control entities but also by the delta.profile, delta.instance.profile, delta_user.profile files which define the path to the current version of Control, C++ Binaries and Java Programs along with their port/host/taskset settings.

Licenses and scripts are located in the delta-bin/config directory and in the deltabin/bin dir. They include the current license files, SSL certificates and the current start/stop scripts.

Kx Control data

Kx Control stores its internal state in kdb+ tables in a directory called tdir, referenced as the DELTACONTROL_TDIR in delta.profile. This contains the current version of all of the entities inside Control and is where Control loads its state from when it starts.

In order to roll back successfully, the environment-configuration and Kx Control data must be reverted to their pre-upgrade state, as follows.

Environment config

Rolling back the environment config involves identifying the backup files and directories created by the install script during the upgrade and using them to revert the current active versions.

Kx Control data

Control tdir data is rolled back using the tdirarchive directory, which contains a TGZ archive of Control’s state, created each time Control is cleanly shut down.

Control qsrc is rolled back using the qsrcarchive directory, which contains TGZ archives of the qsrc files created each time Control is cleanly shut down.

Rollback pre-requisites

To roll back the environment you will need the following;

Upgrade details

To roll back you must identify the last upgrade date and time. If you have the installation logfile (e.g. installDeltaXML.20180301_111047.log) from the upgrade you will be able to see where the script has backed up the Environment Config detailed in the previous section of this document.

The backup location for packages and config changed in version 4.157 of the installDeltaXML.sh script. If you are using an older version of the script you can find the locations from the log file using the following commands:

grep "create backup file" installDeltaXML.20180215_105352.log
create backup file [kxinstall/delta-bin/.bak/delta.profile.20180215_105352]
create backup file [kxinstall/delta-bin/.bak/delta.instance.profile.20180215_105352]
create backup file [kxinstall/delta-bin/.bak/delta_user.profile.20180215_105352]

grep “create backup directory” installDeltaXML.20180215_105352.log
create backup directory [/home/ddempsey/kxinstall/delta-bin/.bak/bin.20180215_105352]

For script version 4.157 and later the locations can be found using the following command:

grep "create backup file" installDeltaXML.20180321_160129.logcreate backup file [
create backup file [kxinstall/backup/20180321_160129/qsrc/qsrc_2250_v1_2018.03.21T160124.tgz]
create backup file [kxinstall/backup/20180321_160129/tdir/tdir_2250_v1_2018.03.21T160124.tgz]
create backup file [kxinstall/backup/20180321_160129/profiles/delta.profile]
create backup file [kxinstall/backup/20180321_160129/profiles/delta.instance.profile]
create backup file [kxinstall/backup/20180321_160129/profiles/delta_user.profile]

For version 4.157 and later the location of the qsrc and tdir files are also given in the output above.

The install timestamp can also be found by looking in delta-bin/.bak (version 4.156 and earlier) and delta-bin/backup (version 4.157 and later).

qsrc archive

The qsrcarchive directory (delta-data/DeltaControlData/qsrcarchive) contains TGZ archives of the qsrc (process templates). The archives are created automatically each time Control is shut down. The process templates differ between versions of Control and Stream so to roll back, identify the archive version which was created just prior to the upgrade.

You can find the location of the previous qsrc archive from the install log (version 1.456 and earlier)

grep "previous qsrc archive" installDeltaXML.20180215_105352.log
previous qsrc archive [qsrc_2250_v2_2018.02.15T105348.tgz]

tdir archive

The tdirarchive directory (delta-data/DeltaControlData/tdirarchive) contains archives of the tdir (control state tables). The archives are created automatically each time Control is shut down. To roll back an upgrade you will also need to roll back the tdir so that all the control entities from Stream and solution packages are reverted.

You can find out the location of the previous tdir archive from the install log (version 1.456 and earlier)

grep "previous tdir archive" installDeltaXML.20180215_105352.log
previous tdir archive [tdir_2250_v2_2018.02.15T105347.tgz]

Tomcat

Web applications (Dashboards/Connect/Analyst etc) are deployed into the Tomcat webapps directory. During an upgrade if Tomcat itself is not being deployed then the previous versions of these webapps are tarred up inside the webapps directory. (Script version 4.146 and earlier.)

ls -1 kxinstall/delta-bin/software/Tomcat_7_0_73/latest/webapps/
analyst
analyst.20180215_130927.tar.gz
connect
connect.20180215_130927.tar.gz
control
control.20180215_130927.tar.gz
dashboards
dashboards.20180215_130927.tar.gz
delta
delta.20180215_130927.tar.gz
ROOT
ROOT.20180215_130927.tar.gz

When rolling back an install you will need to identify the tarballs that were created during the upgrade. You can do this by listing the contents of the webapps directory as above or, if you have the install log from the upgrade, you can get the TGZ names as follows:

grep "tarring up current web application" installDeltaXML.20180215_132727.log
tarring up current web application [ROOT] to [ROOT.20180215_132727.tar.gz] in
[kxinstall/delta-bin/software/Tomcat_7_0_73/latest/webapps]
tarring up current web application [delta] to [delta.20180215_132727.tar.gz] in
[kxinstall/delta-bin/software/Tomcat_7_0_73/latest/webapps]
tarring up current web application [connect] to [connect.20180215_132727.tar.gz] in
[kxinstall/delta-bin/software/Tomcat_7_0_73/latest/webapps]
tarring up current web application [dashboards] to [dashboards.20180215_132727.tar.gz] in
[kxinstall/delta-bin/software/Tomcat_7_0_73/latest/webapps]
tarring up current web application [control] to [control.20180215_132727.tar.gz] in
[kxinstall/delta-bin/software/Tomcat_7_0_73/latest/webapps]

If the upgrade was performed using version 4.147 or later the entire webapps directory is backed up into the backup directory along with the system configurations:

ls -1 backup/20180321_160129/
bin
config
profiles
qsrc
tdir
webapps

If Tomcat itself was upgraded during an upgrade then the new webapps will be deployed into the directory of the new Tomcat. This will remove any manual rollback steps specific to web applications.

Rollback procedures

Single-server deploy

The following section gives the steps required to rollback a platform upgrade using an example where Kx Platform version 4.0.1 has been upgraded to version 4.0.2.

Before proceeding with the rollback, identify the backup files from the following section. For this example the backup files are detailed in the tables below:

Environment config	Backup
delta.profile	kxinstall/backup/<date><time>/profiles/delta.profile
delta_user.profile	kxinstall/backup/<date><time>/profiles/delta_user.profile
delta.instance.profile	kxinstall/backup/<date><time>/profiles/delta.instance.profile
bin	kxinstall/backup/<date><time>/bin

Control Data	Backup
qsrc	kxinstall/backup/<date><time>//qsrc/qsrc_2001_v212_2018.03.13T100617.tgz
tdir	kxinstall/backup/<date><time>/tdir/tdir_2001_v212_2018.03.13T100616.tgz

Web Applications	Backup
webapps	kxinstall/backup/<date><time>/webapps

Steps:

Shut down all running processes.

$ cd kxinstall/delta-bin/bin/
$ ./stopAllProcesses.sh

Shut down Control.

$ cd kxinstall/delta-bin/bin/
$ ./stopDeltaControl.sh

Check with ps for any remaining running processes.

$ ps ux | grep -e "q " -e "java"

Revert environment config from backup.

$ cd kxinstall/delta-bin/
$ cp ../backup/20180313_101225/profiles/delta.profile delta.profile
$ cp ../backup/20180313_101225/profiles/delta_user.profile delta_user.profile
$ cp ../backup/20180313_101225/profiles/delta.instance.profie delta.instance.prfile
$ rm -rf bin/*
$ cp ../backup/20180313_101225/bin/* bin/

Revert Control data from backup.

$ cd kxinstall/delta-data/DeltaControlData
$ mv qsrc qsrc.$(date +%Y%m%d)
$ mv tdir tdir.$(date +%Y%m%d)
$ cd kxinstall/backup/20180313_101225
$ tar -xvzf qsrc/qsrc_2001_v212_2018.03.13T100617.tgz –C /
$ tar -xvzf tdir/tdir_2001_v212_2018.03.13T100616.tgz -C /

Revert Web applications from backup.

This step does not apply if Tomcat was deployed during the upgrade.

$ cd kxinstall/delta-bin/
$ . delta.profile
$ rm -rf $DELTA_WEBAPPS
$ cd kxinstall/backup/20180313_101225
$ cp -r webapps $DELTA_WEBAPPS

Bring system back up.

Control-cluster deploy

The process for rolling back a clustered deploy is almost the same as that used for the single-server deploy in the previous section. The only difference is that the rollback steps have to be executed on each of the hosts in the Control cluster. Backup files need to be identified on all hosts in the Control cluster before proceeding with the rollback.

Ensure all Platform processes are down on all Cluster nodes before performing the rollback steps.

See the Linux Administration section of this guide for more details.