Skip to content

Data Services Events

DAPs and SM provide event callbacks that can be customized with user code. To use the callbacks, the custom code must be installed via a package.

Packaging and Installation

Refer to the User Defined Analytics (UDAs) documentation.

Process and lifecycle events

Syntax: 

.ev.add[eventName;fnName]

The given function is called whenever the event is triggered. The parameters to the function are the event name and payload.

Example:

Code that could be added to sm.q to use the initialization hooks in the SM.

.custom.sm.processInitPreCalled:0b;
.custom.sm.processInitPostCalled:0b;

.custom.sm.processInitPre:{[event;data]
    .custom.sm.processInitPreCalled:1b;
    }

.custom.sm.processInitPost:{[event;data]
    .custom.sm.processInitPostCalled:1b;
    }

.ev.add[`process.init.pre ;`.custom.sm.processInitPre];
.ev.add[`process.init.post;`.custom.sm.processInitPost];

Initialization

Event Component Description Payload
process.init.pre SM,AGG,RC,DAP Before process initialization N/A
process.init.post SM,AGG,RC,DAP After process initialization N/A
process.init.registerAPI DAP Before SG registration so that API's are available via getMeta N/A

Data persistence

Event Component Description Payload
lifecycle.partition.end DAP End of partition (EOI/EOD) received from bus
Deprecates .da.prtnEndCB		 (dict) End of partition message details
signal (symbol): either eoi or eod
lifecycle.reload.pre DAP Before reloading the DB (dict) Reload message details
lifecycle.reload.post DAP After reloading the DB
Deprecates .da.reloadCB		 (dict) Reload message details
lifecycle.eoi.pre SM (EOI) Before EOI processing (dict) Contains date, isEOD and seqid
lifecycle.eoi.post SM (EOI) After EOI processing (dict) Contains date, isEOD and seqid
lifecycle.eod.pre SM (EOD) Before EOD processing (dict) Contains date, nextDate and seq
lifecycle.eod.post SM (EOD) After EOD processing (dict) Contains date, nextDate and seq
lifecycle.batch.pre SM Before batch processing (dict) Contains batchUpdType and request parameters
lifecycle.batch.post SM After batch processing (dict) Contains batchUpdType, request parameters and result

Ingestion events

Define the custom handlers for table ingestion events individually for each table using the following syntax:

.kxdb.addHook[eventName;tableName;fnName]

The given function is called whenever the ingestion event is triggered. The parameters to the function are the table name (symbol) and table data, and the function must return the table data, possibly modified.

The following table describes the available ingestion events:

Event Component Description
data.receive.init SM,DAP Table initialization
data.receive.pre SM,DAP Before stream message processing
data.receive.post SM,DAP After stream message processing

The following example shows code that could be added to sm.q to use the ingestion hooks in the SM:

.kxdb.addHook[`data.receive.pre;;`.custom.sm.receivePre] each `table1`table2`table3; / Register the handler for each table where the ingestion timestamp needs to be recorded

.custom.sm.receivePre:{[tableName;tableData]
    update ingestTime:.z.p from tableData / Replace the ingested column value (likely null) with the current timestamp
    }

For a detailed example of using ingestion hooks to record when the Storage Manager received each row, see the ingestion timestamp walkthrough.

Back to top