RT deduplication publisher clients observability
This API returns information about the publishers connected to the RT and pushing data into a deduplication stream.
To call the REST API, you can use:
Endpoint
The $ENDPOINT variable below is defined as the port-forwarded connection to port 6000 of any sequencer nodes.
bash-4.4$ curl http://$ENDPOINT/dedup-rt-clients
Note
This query can be called directly on a sequencer node.
bash-4.4$ curl http://0:6000/dedup-rt-clients
Endpoint
The $ENDPOINT variable below is defined as port 5001 or 5002 at the hostname of any sequencer nodes.
bash-4.4$ cd /opt/kx/app/rt/replicator/clib/
bash-4,4$ ./rest_proxy --endpoint $ENDPOINT --target dedup-rt-clients
Response
The response includes a response header and a payload.
Each deduplication stream is keyed by its dedup-id and watermark. All the publishers pushing data into a given stream are listed in the publishers section with the metrics of the publisher state regarding the deduplication stream.
The payload members are:
| group | member | type | description |
|---|---|---|---|
| dedup_id | string | The name of a deduplication ID. | |
| watermark | unsigned | The watermark value of the a deduplication stream. Corresponds to the highest id received for this dedup_id |
|
| publishers | key | string | The publishers client name. |
| last_message_merged | datetime | The ts value from the message header of the last merged message for this publisher. |
|
| seq_num_merged | unsigned | The id value from the message header of the last merged message for this publisher. |
|
| msgs_merged | unsigned | Number of messages merged by this publisher into the RT stream. | |
| last_message_pushed | datetime | The ts value from the message header of the last message sent by this publisher. |
|
| seq_num_pushed | unsigned | The id value from the message header of the last message sent by this publisher. |
Note
The details of the RT message header describes the functonality of all the members can be found in the header.
Example
Endpoint
An example of the $ENDPOINT variable is 0:6000.
bash-4.4$ curl http://$ENDPOINT/dedup-rt-clients | jq .
{
"result": [
{
"dedup_id": "banana",
"watermark": 3092,
"publishers": [
{
"pub-3": {
"last_message_merged": "2025-04-29T18:50:18.014684279",
"msgs_merged": 221,
"seq_num_merged": 220,
"last_message_pushed": "2025-04-29T18:55:12.028748276",
"seq_num_pushed": 3074
},
"pub-4": {
"last_message_merged": "2025-04-29T18:55:12.131127231",
"msgs_merged": 2872,
"seq_num_merged": 3092,
"last_message_pushed": "2025-04-29T18:55:12.131127231",
"seq_num_pushed": 3092
}
}
]
},
{
"dedup_id": "vanilla",
"watermark": 3084,
"publishers": [
{
"pub-1": {
"last_message_merged": "2025-04-29T18:50:17.946253038",
"msgs_merged": 220,
"seq_num_merged": 219,
"last_message_pushed": "2025-04-29T18:55:12.152175207",
"seq_num_pushed": 3074
},
"pub-2": {
"last_message_merged": "2025-04-29T18:55:12.161857243",
"msgs_merged": 2865,
"seq_num_merged": 3084,
"last_message_pushed": "2025-04-29T18:55:12.161857243",
"seq_num_pushed": 3084
}
}
]
}
]
}
Note
First, you must exec onto a sequencer node.
bash-4.4$ curl http://0:6000/dedup-rt-clients | jq .
{
"result": [
{
"dedup_id": "banana",
"watermark": 3092,
"publishers": [
{
"pub-3": {
"last_message_merged": "2025-04-29T18:50:18.014684279",
"msgs_merged": 221,
"seq_num_merged": 220,
"last_message_pushed": "2025-04-29T18:55:12.028748276",
"seq_num_pushed": 3074
},
"pub-4": {
"last_message_merged": "2025-04-29T18:55:12.131127231",
"msgs_merged": 2872,
"seq_num_merged": 3092,
"last_message_pushed": "2025-04-29T18:55:12.131127231",
"seq_num_pushed": 3092
}
}
]
},
{
"dedup_id": "vanilla",
"watermark": 3084,
"publishers": [
{
"pub-1": {
"last_message_merged": "2025-04-29T18:50:17.946253038",
"msgs_merged": 220,
"seq_num_merged": 219,
"last_message_pushed": "2025-04-29T18:55:12.152175207",
"seq_num_pushed": 3074
},
"pub-2": {
"last_message_merged": "2025-04-29T18:55:12.161857243",
"msgs_merged": 2865,
"seq_num_merged": 3084,
"last_message_pushed": "2025-04-29T18:55:12.161857243",
"seq_num_pushed": 3084
}
}
]
}
]
}
Endpoint
An example of the $ENDPOINT variable is kxi-mystream-0:5002.
bash-4.4$ cd /opt/kx/app/rt/replicator/clib/
bash-4,4$ ./rest_proxy --endpoint $ENDPOINT --target dedup-rt-clients | jq .
{
"result": [
{
"dedup_id": "banana",
"watermark": 3092,
"publishers": [
{
"pub-3": {
"last_message_merged": "2025-04-29T18:50:18.014684279",
"msgs_merged": 221,
"seq_num_merged": 220,
"last_message_pushed": "2025-04-29T18:55:12.028748276",
"seq_num_pushed": 3074
},
"pub-4": {
"last_message_merged": "2025-04-29T18:55:12.131127231",
"msgs_merged": 2872,
"seq_num_merged": 3092,
"last_message_pushed": "2025-04-29T18:55:12.131127231",
"seq_num_pushed": 3092
}
}
]
},
{
"dedup_id": "vanilla",
"watermark": 3084,
"publishers": [
{
"pub-1": {
"last_message_merged": "2025-04-29T18:50:17.946253038",
"msgs_merged": 220,
"seq_num_merged": 219,
"last_message_pushed": "2025-04-29T18:55:12.152175207",
"seq_num_pushed": 3074
},
"pub-2": {
"last_message_merged": "2025-04-29T18:55:12.161857243",
"msgs_merged": 2865,
"seq_num_merged": 3084,
"last_message_pushed": "2025-04-29T18:55:12.161857243",
"seq_num_pushed": 3084
}
}
]
}
]
}