RT deduplication publisher clients observability
This API returns information about the publishers connected to the RT and pushing data into a deduplication stream.
To call the REST API, you can use:
Endpoint
The $ENDPOINT
variable below is defined as the port-forwarded connection to port 6000 of any sequencer nodes.
bash-4.4$ curl http://$ENDPOINT/dedup-rt-clients
Note
This query can be called directly on a sequencer node.
bash-4.4$ curl http://0:6000/dedup-rt-clients
Endpoint
The $ENDPOINT
variable below is defined as port 5001 or 5002 at the hostname of any sequencer nodes.
bash-4.4$ cd /opt/kx/app/rt/replicator/clib/
bash-4,4$ ./rest_proxy --endpoint $ENDPOINT --target dedup-rt-clients
Response
The response includes a response header and a payload.
Each deduplication stream is keyed by its dedup-id and watermark. All the publishers pushing data into a given stream are listed in the publishers section with the metrics of the publisher state regarding the deduplication stream.
The payload members are:
group | member | type | description |
---|---|---|---|
dedup_id | string | The name of a deduplication stream. | |
watermark | unsigned | The watermark value of the a deduplication stream. | |
publishers | key | string | The publisher's client name. |
last_message | datetime | The ts member from the message header of the lastly pushed message. | |
msgs_merged | unsigned | Number of messages are merged by this publisher into the subscriber stream. | |
seq_num | unsigned | The id member from the message header of the lastly pushed message. |
Note
The details of the RT message header describes the functonality of all the members can be found in the header.
Example
Endpoint
An example of the $ENDPOINT
variable is 0:6000
.
bash-4.4$ curl http://$ENDPOINT/dedup-rt-clients | jq .
{
"result": [
{
"dedup_id": "banana",
"watermark": 103,
"publishers": [
{
"pub-3.kxi-mystream": {
"last_message": "2025-01-17T09:58:47.510844057",
"msgs_merged": 104,
"seq_num": 103
},
"pub-4.kxi-mystream": {
"last_message": "2025-01-17T09:58:47.359598117",
"msgs_merged": 0,
"seq_num": 87
}
}
]
},
{
"dedup_id": "vanilla",
"watermark": 103,
"publishers": [
{
"pub-1.kxi-mystream": {
"last_message": "2025-01-17T09:58:46.286049652",
"msgs_merged": 57,
"seq_num": 71
},
"pub-2.kxi-mystream": {
"last_message": "2025-01-17T09:58:47.924897920",
"msgs_merged": 47,
"seq_num": 103
}
}
]
}
]
}
Note
First, you must exec onto a sequencer node.
bash-4.4$ curl http://0:6000/dedup-rt-clients | jq .
{
"result": [
{
"dedup_id": "banana",
"watermark": 103,
"publishers": [
{
"pub-3.kxi-mystream": {
"last_message": "2025-01-17T09:58:47.510844057",
"msgs_merged": 104,
"seq_num": 103
},
"pub-4.kxi-mystream": {
"last_message": "2025-01-17T09:58:47.359598117",
"msgs_merged": 0,
"seq_num": 87
}
}
]
},
{
"dedup_id": "vanilla",
"watermark": 103,
"publishers": [
{
"pub-1.kxi-mystream": {
"last_message": "2025-01-17T09:58:46.286049652",
"msgs_merged": 57,
"seq_num": 71
},
"pub-2.kxi-mystream": {
"last_message": "2025-01-17T09:58:47.924897920",
"msgs_merged": 47,
"seq_num": 103
}
}
]
}
]
}
Endpoint
An example of the $ENDPOINT
variable is kxi-mystream-0:5002
.
bash-4.4$ cd /opt/kx/app/rt/replicator/clib/
bash-4,4$ ./rest_proxy --endpoint $ENDPOINT --target dedup-rt-clients | jq .
{
"result": [
{
"dedup_id": "banana",
"watermark": 103,
"publishers": [
{
"pub-3.kxi-mystream": {
"last_message": "2025-01-17T09:58:47.510844057",
"msgs_merged": 104,
"seq_num": 103
},
"pub-4.kxi-mystream": {
"last_message": "2025-01-17T09:58:47.359598117",
"msgs_merged": 0,
"seq_num": 87
}
}
]
},
{
"dedup_id": "vanilla",
"watermark": 103,
"publishers": [
{
"pub-1.kxi-mystream": {
"last_message": "2025-01-17T09:58:46.286049652",
"msgs_merged": 57,
"seq_num": 71
},
"pub-2.kxi-mystream": {
"last_message": "2025-01-17T09:58:47.924897920",
"msgs_merged": 47,
"seq_num": 103
}
}
]
}
]
}