Troubleshooting runtime errors¶
Runtime errors can occur within the Monitoring components. Here are common resolution steps.
Daemon¶
DC registration error InvalidWorkflowForGilterException¶
This error occurs when .mon.cfg.onlyMonitoringProcessesInWorkflows is set and one or more of the workflows specified in this list is not a valid workflow in the currently connected Delta Control.
Resolution Review the list of workflows in .mon.cfg.onlyMonitoringProcessesInWorkflows and ensure that they match valid workflows in the current Delta Control.
DC registration error NotEnoughConnectionsForMonitoringException¶
This error occurs when there are more processes to monitor than the daemon is able to create based on operating-system limitations.
Resolution 1 Reduce the list of processes to be monitored by filtering by workflow
Resolution 2 Increase the maximum number of open files available to the Monitoring daemon component. This can be done by adding the configuration LimitNOFILE=*new-max* into the daemon's systemd file:
[Service]
LimitNOFILE=4096
Monitored process disconnect/reconnect loop¶
The daemon does not maintain a connection with any process to be monitored that does not successfully register with it. If the registration fails, the daemon will disconnect from it. However, the timer job to check for down processes will attempt connection again at the next interval.
For reference, the lifecycle of the daemon connecting and registering to a process is:
- Connect to process
- Send asynchronous registration request
- Wait for
.mon.cfg.registrationMaxWaitTimefor an asynchronous response back from the process. If valid response returned, process registration is successful; otherwise process registration fails and the process is disconnected.
Resolution Ensure that any custom processes have the Monitoring client and common metric libraries loaded and initialized correctly:
/ Monitoring 2.0 - connectivity and common metrics
.al.loadinstruction each `.daas.cmon.conn`.daas.cmon.common
.daas.cmon.init[]
.daas.cmon.common.init[]