Monitoring your system

The Monitoring Framework is available in Refinery 4.3.2 or later.

Monitoring requires additional installation and configuration alongside the standard Refinery application.

The Monitoring Framework allows integration with external platform-monitoring systems to access Refinery's state. All monitoring components have a common application prefix - kx-refinery-monitoring-.

Monitoring features

Monitoring within Refinery provides the following features:

  • Lightweight components requiring low CPU and RAM usage
  • Components reside externally from the Refinery application
  • Monitoring information is captured and distributed in real-time
  • Monitoring information can be captured back into Refinery for querying and visualization
  • Custom monitoring data can be configured
  • Standard RHEL/CentOS RPM installation mechanism
  • Components are managed along with their dependencies via systemd

Prerequisites

kdb+ must be [installed]code.kx.com/q/learn/install/):

  • the q binary must be within $QHOME/l64 or $QHOME/l32
  • $QLIC must be set if the k4.lic or kc.lic file is not in $QHOME

Version map

Monitoring components are independent of Refinery and have a separate release history (see below).

Refinery version Monitoring version
4.2.x 0.2.5
4.3.x 0.3.6
4.4.x 0.5.0
4.5.x 0.6.0
5.0.x or later 0.8.2
5.7.5 0.8.2
5.8.2 0.8.3

More detail related to upgrading between versions can be found in the Upgrade Notes.

Component overview

Monitoring has three types of components.

Component type Name Function
Refinery Monitoring demon kx-refinery-monitoring-daemon The destination for all Refinery processes to send monitoring information
Refinery Monitoring API kx-refinery-monitoring-api-* Provide data access to the Monitoring data via a number of different methods
Refinery metric generators kx-refinery-monitoring-gen-* Generates additional metrics for monitoring alongside metrics generated by the Refinery application itself

The following components are available.

name                   type           ID                              initial version 
--------------------------------------------------------------------------------------
Monitoring Daemon      Daemon         kx-refinery-monitoring-daemon   0.2.5-2019.08.23
Web API                API            kx-refinery-monitoring-api-web  0.2.5-2019.08.23
Elasticsearch Uploader API            kx-refinery-monitoring-api-es   0.2.5-2019.08.23
Alerting               API            kx-refinery-monitoring-api-alrt 0.5.0-2020.03.03
System Info Generator  Generator      kx-refinery-monitoring-gen-sysi 0.3.4-2019.11.08

Monitoring process layout

The recommended architecture for running the Monitoring components is as follows:

  • Monitoring daemon per Refinery server
  • API component per Monitoring daemon
  • System Information generator per server, if not gathered separately by another monitoring system

Components

kx-refinery-monitoring-daemon

The Monitoring daemon is the component that connects to all Refinery processes and acts as the endpoint for monitoring information ('metrics') to be sent to.

The daemon also publishes all information receives to downstream subscribers. This can be one of the API components or, optionally, a feed handler within Refinery itself to allow the capture and persistence of all the data.

kx-refinery-monitoring-api-web

The Web API provides a HTTP JSON and WebSocket API on top of the Monitoring data published by the daemon.

kx-refinery-monitoring-api-es

The Elasticsearch Uploader uploads all Monitoring data into Elasticsearch for capturing and analysis with Kibana.

kx-refinery-monitoring-api-alrt

The Alerting component provides email and Slack alerting based on defined thresholds.

kx-refinery-monitoring-gen-sysi

The System Information Generator publishes system-level and process-level metrics gathered directly from the operating system itself (similar to SIGAR).