Roll back an upgrade
This guide details the procedure for rolling back the Kx Platform after an upgrade. Please note that when an upgrade fails at deploy time the rollback is handled automatically by the install script. This guide focuses on the steps required to roll back an upgraded system to a previously installed version.
How the rollback works
The Kx Platform consists of three different types of content:
- Data
-
The data generated by the system.
The rollback procedure does not cover rolling back data generated by the system since the upgrade.
This is solution-specific and outside of the scope of this document.
- Environment config
-
Applications and binaries are configured via Control entities but also by the
delta.profile
,delta.instance.profile
,delta_user.profile
files which define the path to the current version of Control, C++ Binaries and Java Programs along with their port/host/taskset settings. -
Licenses and scripts are located in the
delta-bin/config
directory and in thedeltabin/bin
dir. They include the current license files, SSL certificates and the current start/stop scripts. - Kx Control data
-
Kx Control stores its internal state in kdb+ tables in a directory called
tdir
, referenced as theDELTACONTROL_TDIR
indelta.profile
. This contains the current version of all of the entities inside Control and is where Control loads its state from when it starts.
In order to roll back successfully, the environment-configuration and Kx Control data must be reverted to their pre-upgrade state, as follows.
Environment config
Rolling back the environment config involves identifying the backup files and directories created by the install script during the upgrade and using them to revert the current active versions.
Kx Control data
Control tdir
data is rolled back using the tdirarchive
directory, which contains a TGZ archive of Control’s state, created each time Control is cleanly shut down.
Control qsrc
is rolled back using the qsrcarchive
directory, which contains TGZ archives of the qsrc
files created each time Control is cleanly shut down.
Rollback pre-requisites
To roll back the environment you will need the following;
Upgrade details
To roll back you must identify the last upgrade date and time. If you have the installation logfile (e.g. installDeltaXML.20180301_111047.log) from the upgrade you will be able to see where the script has backed up the Environment Config detailed in the previous section of this document.
The backup location for packages and config changed in version 4.157 of the installDeltaXML.sh script. If you are using an older version of the script you can find the locations from the log file using the following commands:
grep "create backup file" installDeltaXML.20180215_105352.log
create backup file [kxinstall/delta-bin/.bak/delta.profile.20180215_105352]
create backup file [kxinstall/delta-bin/.bak/delta.instance.profile.20180215_105352]
create backup file [kxinstall/delta-bin/.bak/delta_user.profile.20180215_105352]
grep “create backup directory” installDeltaXML.20180215_105352.log
create backup directory [/home/ddempsey/kxinstall/delta-bin/.bak/bin.20180215_105352]
For script version 4.157 and later the locations can be found using the following command:
grep "create backup file" installDeltaXML.20180321_160129.logcreate backup file [
create backup file [kxinstall/backup/20180321_160129/qsrc/qsrc_2250_v1_2018.03.21T160124.tgz]
create backup file [kxinstall/backup/20180321_160129/tdir/tdir_2250_v1_2018.03.21T160124.tgz]
create backup file [kxinstall/backup/20180321_160129/profiles/delta.profile]
create backup file [kxinstall/backup/20180321_160129/profiles/delta.instance.profile]
create backup file [kxinstall/backup/20180321_160129/profiles/delta_user.profile]
For version 4.157 and later the location of the qsrc
and tdir
files are also given in the output above.
The install timestamp can also be found by looking in delta-bin/.bak
(version 4.156 and earlier) and delta-bin/backup
(version 4.157 and later).
qsrc
archive
The qsrcarchive
directory (delta-data/DeltaControlData/qsrcarchive
) contains TGZ archives of the qsrc
(process templates). The archives are created automatically each time Control is shut down. The process templates differ between versions of Control and Stream so to roll back, identify the archive version which was created just prior to the upgrade.
You can find the location of the previous qsrc
archive from the install log (version 1.456 and earlier)
grep "previous qsrc archive" installDeltaXML.20180215_105352.log
previous qsrc archive [qsrc_2250_v2_2018.02.15T105348.tgz]
tdir
archive
The tdirarchive
directory (delta-data/DeltaControlData/tdirarchive
) contains archives of the tdir
(control state tables). The archives are created automatically each time Control is shut down. To roll back an upgrade you will also need to roll back the tdir
so that all the control entities from Stream and solution packages are reverted.
You can find out the location of the previous tdir
archive from the install log (version 1.456 and earlier)
grep "previous tdir archive" installDeltaXML.20180215_105352.log
previous tdir archive [tdir_2250_v2_2018.02.15T105347.tgz]
Tomcat
Web applications (Dashboards/Connect/Analyst etc) are deployed into the Tomcat webapps
directory. During an upgrade if Tomcat itself is not being deployed then the previous versions of these webapps are tarred up inside the webapps directory. (Script version 4.146 and earlier.)
ls -1 kxinstall/delta-bin/software/Tomcat_7_0_73/latest/webapps/
analyst
analyst.20180215_130927.tar.gz
connect
connect.20180215_130927.tar.gz
control
control.20180215_130927.tar.gz
dashboards
dashboards.20180215_130927.tar.gz
delta
delta.20180215_130927.tar.gz
ROOT
ROOT.20180215_130927.tar.gz
When rolling back an install you will need to identify the tarballs that were created during the upgrade. You can do this by listing the contents of the webapps
directory as above or, if you have the install log from the upgrade, you can get the TGZ names as follows:
grep "tarring up current web application" installDeltaXML.20180215_132727.log
tarring up current web application [ROOT] to [ROOT.20180215_132727.tar.gz] in
[kxinstall/delta-bin/software/Tomcat_7_0_73/latest/webapps]
tarring up current web application [delta] to [delta.20180215_132727.tar.gz] in
[kxinstall/delta-bin/software/Tomcat_7_0_73/latest/webapps]
tarring up current web application [connect] to [connect.20180215_132727.tar.gz] in
[kxinstall/delta-bin/software/Tomcat_7_0_73/latest/webapps]
tarring up current web application [dashboards] to [dashboards.20180215_132727.tar.gz] in
[kxinstall/delta-bin/software/Tomcat_7_0_73/latest/webapps]
tarring up current web application [control] to [control.20180215_132727.tar.gz] in
[kxinstall/delta-bin/software/Tomcat_7_0_73/latest/webapps]
If the upgrade was performed using version 4.147 or later the entire webapps
directory is backed up into the backup
directory along with the system configurations:
ls -1 backup/20180321_160129/
bin
config
profiles
qsrc
tdir
webapps
If Tomcat itself was upgraded during an upgrade then the new webapps
will be deployed into the directory of the new Tomcat. This will remove any manual rollback steps specific to web applications.
Rollback procedures
Single-server deploy
The following section gives the steps required to rollback a platform upgrade using an example where Kx Platform version 4.0.1 has been upgraded to version 4.0.2.
Before proceeding with the rollback, identify the backup files from the following section. For this example the backup files are detailed in the tables below:
Environment config | Backup |
---|---|
delta.profile | kxinstall/backup/<date><time>/profiles/delta.profile |
delta_user.profile | kxinstall/backup/<date><time>/profiles/delta_user.profile |
delta.instance.profile | kxinstall/backup/<date><time>/profiles/delta.instance.profile |
bin | kxinstall/backup/<date><time>/bin |
Control Data | Backup |
---|---|
qsrc | kxinstall/backup/<date><time>//qsrc/qsrc_2001_v212_2018.03.13T100617.tgz |
tdir | kxinstall/backup/<date><time>/tdir/tdir_2001_v212_2018.03.13T100616.tgz |
Web Applications | Backup |
---|---|
webapps | kxinstall/backup/<date><time>/webapps |
Steps:
Shut down all running processes.
$ cd kxinstall/delta-bin/bin/
$ ./stopAllProcesses.sh
Shut down Control.
$ cd kxinstall/delta-bin/bin/
$ ./stopDeltaControl.sh
Check with ps
for any remaining running processes.
$ ps ux | grep -e "q " -e "java"
Revert environment config from backup.
$ cd kxinstall/delta-bin/
$ cp ../backup/20180313_101225/profiles/delta.profile delta.profile
$ cp ../backup/20180313_101225/profiles/delta_user.profile delta_user.profile
$ cp ../backup/20180313_101225/profiles/delta.instance.profie delta.instance.prfile
$ rm -rf bin/*
$ cp ../backup/20180313_101225/bin/* bin/
Revert Control data from backup.
$ cd kxinstall/delta-data/DeltaControlData
$ mv qsrc qsrc.$(date +%Y%m%d)
$ mv tdir tdir.$(date +%Y%m%d)
$ cd kxinstall/backup/20180313_101225
$ tar -xvzf qsrc/qsrc_2001_v212_2018.03.13T100617.tgz –C /
$ tar -xvzf tdir/tdir_2001_v212_2018.03.13T100616.tgz -C /
Revert Web applications from backup.
This step does not apply if Tomcat was deployed during the upgrade.
$ cd kxinstall/delta-bin/
$ . delta.profile
$ rm -rf $DELTA_WEBAPPS
$ cd kxinstall/backup/20180313_101225
$ cp -r webapps $DELTA_WEBAPPS
Bring system back up.
Control-cluster deploy
The process for rolling back a clustered deploy is almost the same as that used for the single-server deploy in the previous section. The only difference is that the rollback steps have to be executed on each of the hosts in the Control cluster. Backup files need to be identified on all hosts in the Control cluster before proceeding with the rollback.
Ensure all Platform processes are down on all Cluster nodes before performing the rollback steps.
See the Linux Administration section of this guide for more details.