KX Sensors application architecture¶
This page gives a high-level overview of how data flows through KX Sensors, from client application to storage and back, in an RT-only node configuration.
A client application interacts with KX Sensors in two ways: it publishes raw sensor readings on a data ingestion stream, and it sends and receives API requests and responses on a separate stream.
On the sensor nodes, the data ingestion stream feeds an external EMS stream for raw messages. From there:
- SDL (Sensor Data Loader) ingests the raw messages, applies any required transformation, and publishes formatted messages to an internal EMS stream for formatted messages.
- From the internal stream, a stream processor handles MDL, SDL, VEE, CDL, and Agg processing, and DBW (Database Writer) writes data down to disk, first to an IDB (Intraday Database) and ultimately to an HDB (Historical Database).
API requests from the client are handled by GW (Gateway), which determines which data services are needed to fulfill the request and directs it to an RP (Response Process). RP queries the appropriate combination of RDB (Real-Time Database), IDB, and HDB, and returns the aggregated result to the client as an API response.
A separate RT-only node runs etcd, RTO (RT Orchestrator), and Reliable Transport only, without the other sensor node processes.