Skip to content

Overview of dynamic upgrades

This page explains how dynamic and offline upgrades work in KX Sensors, when to use each, and what to expect in a multi-node environment.

This chapter covers dynamic and offline upgrades. Dynamic upgrades allow you to upgrade any component of your system without the need for a system shutdown and restart.

Dynamic upgrades are performed on the fly with zero downtime or disruption to KX Sensors operation. You can use dynamic upgrades in the following situations:

  • To install a minor version upgrade or patch release from KX.
  • To make minor non-KX changes to your system such as adding or removing table columns.

Note

If you are making major non-KX changes to your system, you may need to perform an offline upgrade with a system or node restart. Consult with KX when uncertain and always ensure upgrades are tested in a UAT environment before production release.

Dynamic upgrades fully support partial upgrades targeting service classes or processes. You can additionally target individual nodes and deploy schema or code changes only to that node as a rolling upgrade. See Multi-node environments for how this works across a cluster.

If you make a mistake during a dynamic upgrade, you can roll back or downgrade the system to a previous version and then re-deploy the corrected package.

Fixes applied ahead of a patch release (hotfixes) must also be delivered as versioned packages and applied with a dynamic upgrade, rather than by editing deployed files. See Create a hotfix.

Typical upgrade steps

  1. Create a new package containing your schema, code, or API changes in your test environment. The package name must be distinct from existing packages already deployed to the system. For example, increment kxs-core-3.0.0 to kxs-core-3.0.1.
  2. Update .pkg to increment the version number associated with the package.
  3. Copy the package to the package directory on all nodes involved in the upgrade.
  4. Perform the upgrade using the KX Sensors CLI, which generates a unique revision number and propagates the upgrade to all specified processes.

For the full procedure, with commands, verification and rollback, see Perform a dynamic upgrade.

Warning

Dynamic upgrade of the config objects feed.yaml and ems.yaml is not fully supported. These are typically configured in the env/ package. Any upgrade of the env/ package must not include changes to these objects.

Database integrity

Dynamic and offline upgrades involving a schema change must never be performed during DBC (Database Copy) operations.

Multi-node environments

In a multi-node environment, a release may be performed in a rolling manner of single-node releases, simultaneous multi-node releases, or a combination of both. Nodes not involved in the release continue to run on the older version. In all cases, nodes at different schema levels convert messages targeted to each other to the format that they expect in-transit.

If a release is issued while a node or process is down, it starts at its assigned revision when it comes back up. This is why every package in the directory list must exist on every node, including nodes that are down at the time of the release.

How changes propagate

sequenceDiagram
    participant Op as Operator (kxsctl)
    participant Reg as Process version registry
    participant P as Each targeted process
    Op->>Op: Pre-DU validation on each target node
    Op->>Reg: Record deployment (directory list + file hashes) and target revision
    Reg-->>P: Registry change notification
    P->>P: Compute changed files between current and target revision
    P->>P: UPGRADING: schema, q code, APIs, configuration
    P->>P: Back to ACTIVE at target revision
    Op->>Op: Post-DU: refresh package links on each target node
  • Packages are copied by you. kxsctl does not distribute packages. Every package directory in the directory list must exist in the package directory of each target node, and of any node you may target later. Pre-DU validation checks that the directories exist on each target node (kxsctl deploy validate --dirs), but does not copy them.
  • Every node needs an identical copy. The node where you run kxsctl deploy records the file hashes for the new deployment. If a package directory differs between nodes, the DU may reload the wrong files on some nodes. Copy each package from a single source and don't edit the copies.
  • The revision is pushed through the registry. kxsctl deploy set and kxsctl deploy apply record the deployment and the targets' revision in the process version registry, which is shared by all nodes. Each process watches the registry and upgrades itself independently when its assigned revision changes. You can run kxsctl deploy from any node, whichever nodes are targeted.
  • Only changed files are reloaded. Each process compares the file hashes of its current and target revisions, then reloads the changed schema, q code (for files it loads), APIs and configuration objects, in that order. A process that is not yet ACTIVE queues the upgrade until it is.
  • Configuration changes are reloaded, not necessarily acted on. After a DU, the configuration cache is cleared and listeners on the changed objects are notified. Code that reads the object (for example with .cfg.get) sees the new value. Code that copied a value at startup, or a parameter that is only read at startup, needs a restart of the process to take effect.
  • Package links are refreshed. After the registry is updated, kxsctl repoints the unversioned package links (for example, example -> example-1.0.1) on each target node, using the pkg.name and pkg.version in each package's .pkg file.

Dynamic vs offline upgrades

Dynamic upgrades differ from offline upgrades in that offline upgrades require the system to be restarted. In most cases, prefer dynamic upgrades. Notable exceptions are listed in Perform an offline upgrade and typically involve a custom upgrade script for schema modifications or changes to non-trivial configuration.

Next steps