Skip to content

Start and stop the system

This page covers the prerequisite step of starting etcd, along with starting a fully offline KX Sensors system and safely stopping a running one.

Start etcd

You must start etcd before you can perform any node-related operations such as adding, removing, starting or stopping a node. Run the following command to find out whether etcd is running in your environment:

systemctl status etcd

Start an offline system

Starting an offline system will start all necessary KX Sensors processes, including the RT processes started by RTO.

  1. Make sure that etcd is installed and running.

  2. Open a command line terminal to one of the KX Sensors nodes and enter the following command:

    kxsctl start <node>

  3. Start any client-side applications after the startup process is complete and validated.

Validate system startup

It is important to confirm that the system has properly started and that it is ready to perform its intended functions. The following are the key areas to assess to confirm a system's successful startup.

Check that all expected processes are running.

Check the process log files for any errors.

Check carefully the DBW log file for any issues. Make sure that every process' log replay has run successfully without any errors indicated in the logs.

Upon a system start, DBW will undergo the most work to either initialize or update the database to the real-time state. Once this procedure is completed, downstream processes can start. Because of this it is crucial that DBW starts without any errors.

Log replay would have only occurred if the system has been running prior to this startup. Log replay start and completion is indicated by log messages Starting log replay and Log replay complete.

Any ERROR or FATAL messages may indicate critical issues during the startup procedure as well as any stack trace log messages, which are usually preceded by Stack:.

Check RDB, IDB and HDB for in-memory tables. If there was recent data ingestion before the system restart, RDB should have some data for basic, splay, part category of tables. IDB and HDB should have basic and splay categories in-memory. The descriptions of these table categories can be found in Table types.

Check the IDB and HDB to ensure that the on-disk DB has been mounted properly by issuing queries that involve partitioned tables.

After confirming that the log-replay for DBW has finished, check the HDB and IDB directories to confirm that there is only one link directory (i.e. one of hdb.0 or hdb.1 and one of idb.0 and idb.1). These directories need not match; in other words, having hdb.0 and idb.1 is acceptable.

If this startup occurred after a system failure, check the process log file for the failed process(es). The last few lines of the log file should say what the process was doing just before it was shut down. If not apparent, check the log files for other DB-related processes in the following order: DBW, IDB, HDB, DBM.

Stop an online system

Stopping an online system will stop all KX Sensors processes, including the RT processes started by RTO.

Note

etcd is considered a pre-requisite service and is not terminated by this procedure.

  1. Perform any necessary client-side shutdowns, like data feeds, before beginning the shutdown of KX Sensors.

  2. Use the KX Sensors CLI to stop the system.

    kxsctl stop <node>

Next steps