About systemd¶
This page describes how systemd unit files govern KX Sensors process restart policies.
systemd unit files are used to ensure processes are restarted if they shut down unexpectedly. The restart policy of processes is governed by the StartLimitBurst and StartLimitIntervalSec settings. For etcd and qetcd, the default settings are:
StartLimitBurst=5
StartLimitIntervalSec=60s
This means if systemd exhausts 5 attempts to start etcd within 60 seconds, it no longer attempts to restart it.
For all other KXS processes, the default settings are:
StartLimitBurst=5
StartLimitIntervalSec=120s
A longer interval is given for other KXS processes to account for long-running startup activities, for example an RDB replaying an RT stream.
Changes to these settings can be made to the unit files directly. One exists for each process. Look up the location of a unit file using systemctl status:
[root@kxsa logs]# systemctl status kxsDBW_A.service
● kxsDBW_A.service - KX Sensors Service: kxsDBW_A
Loaded: loaded (/etc/systemd/system/kxsDBW_A.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Tue 2025-01-21 09:26:43 UTC; 26min ago
Process: 4603 ExecStart=/root/bin/q/l64/q /root/kxs/packages/kxs-core/src/proc/common/boot.q -name kxsDBW_A -sc DBW -domain A -port 20007 -seq 0 -s 20 -E 0 (code=exited, status=1/FAILU> Main PID: 4603 (code=exited, status=1/FAILURE)
Jan 21 09:26:43 kxsa.firstderivatives.com systemd[1]: kxsDBW_A.service: Service RestartSec=1s expired, scheduling restart.
Jan 21 09:26:43 kxsa.firstderivatives.com systemd[1]: kxsDBW_A.service: Scheduled restart job, restart counter is at 6.
Jan 21 09:26:43 kxsa.firstderivatives.com systemd[1]: Stopped KX Sensors Service: kxsDBW_A.
Jan 21 09:26:43 kxsa.firstderivatives.com systemd[1]: kxsDBW_A.service: Start request repeated too quickly.
Jan 21 09:26:43 kxsa.firstderivatives.com systemd[1]: kxsDBW_A.service: Failed with result 'exit-code'.
Jan 21 09:26:43 kxsa.firstderivatives.com systemd[1]: Failed to start KX Sensors Service: kxsDBW_A.
[root@kxsa logs]# ls /etc/systemd/system/kxsDBW_A.service
/etc/systemd/system/kxsDBW_A.service
systemd settings can also be set at a system-wide level. To do so, update /etc/systemd/system.conf or /etc/systemd/user.conf. The former is used if the systemd user is root; otherwise, update /etc/systemd/user.conf. Changes to these settings require a systemd daemon reload and a restart of the systemd service (that is, kxsctl restart <process>).
Regenerate unit files¶
A KX Sensors release can change the unit file templates, and a manifest change can add or remove processes. To bring the unit files on a node into line with the current templates and manifest, run kxsctl refresh:
[root@kxsa logs]# kxsctl refresh A
Add --prune to also stop processes that are no longer in the manifest and remove their unit files. Changes you made to a unit file by hand are lost when the file is regenerated.
DefaultLimitNOFILE is an important systemd setting for database processes that may open many files while traversing a database during a query. If this limit isn't set high enough, you may encounter "Too many open files" errors during query execution.
[root@kxsa logs]# tail -1 /etc/systemd/system.conf
DefaultLimitNOFILE=655355
[root@kxsa logs]# systemctl daemon-reload
[root@kxsa logs]# kxsctl restart HDB
To confirm the limit has been updated, connect to an HDB and check ulimit -n. The following example is run from a maintenance console:
q)ohp[`kxsHDB_A1]"\\ulimit -n"
2025.07.31 08:16:27.415 INFO [con] COMM Opening sync connection `kxsHDB_A1
"655355"