Testing a UDA

Thorough testing and debugging of UDAs is essential to ensure correct behavior. This page provides a step-by-step guide to testing and debugging the query and aggregation functions within a UDA, helping you identify and resolve issues to ensure accurate and efficient data processing.

This page covers the following sections:

Testing query functions
Testing aggregation functions

kdb Insights provides an iterative approach to UDA development by allowing you to visualize execution output and verify that the results are valid and return the expected data. The extension allows you to connect to a specific DAP process and execute code directly from the editor. By sending queries to the same DAP each time, you ensure that your UDA code is executed in a consistent run-time environment, which helps avoid differences in state or configuration across replicas. It also ensures persistence of variables between test executions.

We recommend you add logging to your UDA functions to assist with testing a debugging. Refer to Adding logging to UDAs for more details.

When creating a query function, you can use the following methods to test and iterate on the code using the kdb VS Code Extension.

Add a Connection to the kdb Insights Enterprise deployment.
1. Select the CONNECTIONS panel and either:
  - Click Add Connection
  - Click +
  Note
  
  To add a connection you need a folder open in VS Code.
2. Add a My q connection
Right click on the new connection in the CONNECTIONS panel and select Connect server.
Create a new file of type:
- Source file or KX Notebook
- Select the appropriate connection from the Connection dropdown

Enable detailed debugging

To pinpoint the exact location of the failure, re-run the query while connected to the process with error trapping enabled. This approach generates more detailed debugging output.

Review the detailed output:

q)2024-08-12 15:51:51.154 [hdb] DEBUG KXCTX {ba62e0b9-f290-4abd-be3d-dcc91cb6116d} Starting context, corr=ba62e0b9-f290-4abd-be3d-dcc91cb6116d
2024-08-12 15:51:51.154 [hdb] DEBUG KXCTX {ba62e0b9-f290-4abd-be3d-dcc91cb6116d} Setting context to no-save, reason='requests not saved in DAP'
2024-08-12 15:51:51.154 [hdb] DEBUG DA {ba62e0b9-f290-4abd-be3d-dcc91cb6116d} Executing .custom.countBy
type error
.custom.countBy[0]   {[table;startTS;endTS;byCols]1+`}
                                                ^
q))table / See what current table argument is
`trade
q))startTS / See what current startTS argument is
-0Wp

Based on the output you can see specifically where the function is failing. Additionally, you can inspect the current values of the arguments. For larger functions, you can also run through the code line by line to identify the underlying issue.

Getting the backtrace of query or aggregations functions remotely

Set the environment variable KXI_DEBUG_QUERY to true on the resource-coordinators, or set debug to 1b in the opts of a request. When you do, if there is a failure in either a query function or an aggregation function, then the backtrace is returned in the header in the key bt.

Example with qipc:

q)GATEWAY:10i / Handle to gateway
q)args:enlist[`table]!enlist `trade
q)r:GATEWAY (`.kxi.qsql;enlist[`query]!enlist"1+`";`ignored;enlist[`debug]!enlist 1b) / The ignored is the callback function in async case
q)-1 r[0;`bt]
{}[0]   {1+`}
...

Testing query functions

Run the query and analyze the logs

If a query encounters an error but continues execution, the DAP logs will contain details of the issue.

For example, consider the following query function, which generates a type error:

.custom.countBy:{[table;startTS;endTS;byCols]
    1+`
    }

After calling this API, the DAP logs provide the following output:

2024-08-12 15:45:30.146 [hdb] DEBUG KXCTX {23329698-9642-4654-b65e-81bfefe16b05} Starting context, corr=23329698-9642-4654-b65e-81bfefe16b05
2024-08-12 15:45:30.146 [hdb] DEBUG KXCTX {23329698-9642-4654-b65e-81bfefe16b05} Setting context to no-save, reason='requests not saved in DAP'
2024-08-12 15:45:30.146 [hdb] DEBUG DA {23329698-9642-4654-b65e-81bfefe16b05} Executing .custom.countBy
2024-08-12 15:45:30.146 [hdb] ERROR SAPI {23329698-9642-4654-b65e-81bfefe16b05} Error (type) encountered executing .custom.countBy, rc=6 ac=11 ai=Unexpected error (type) encountered executing .custom.countBy
2024-08-12 15:45:30.146 [hdb] DEBUG DA {23329698-9642-4654-b65e-81bfefe16b05} Completed .custom.countBy, rc=6 ac=11 ai=Unexpected error (type) encountered executing .custom.countBy
2024-08-12 15:45:30.146 [hdb] DEBUG DA {23329698-9642-4654-b65e-81bfefe16b05} Sending response to aggregator, agg=:10.244.0.9:5070
2024-08-12 15:45:30.147 [hdb] DEBUG KXCTX {23329698-9642-4654-b65e-81bfefe16b05} Ending context, no-save

The log indicates a type error, which helps identify that the issue is related to data types.

Enable detailed debugging

To pinpoint the exact location of the failure, re-run the query while connected to the process with error trapping enabled. This approach generates more detailed debugging output.

Review the detailed output:

q)2024-08-12 15:51:51.154 [hdb] DEBUG KXCTX {ba62e0b9-f290-4abd-be3d-dcc91cb6116d} Starting context, corr=ba62e0b9-f290-4abd-be3d-dcc91cb6116d
2024-08-12 15:51:51.154 [hdb] DEBUG KXCTX {ba62e0b9-f290-4abd-be3d-dcc91cb6116d} Setting context to no-save, reason='requests not saved in DAP'
2024-08-12 15:51:51.154 [hdb] DEBUG DA {ba62e0b9-f290-4abd-be3d-dcc91cb6116d} Executing .custom.countBy
type error
.custom.countBy[0]   {[table;startTS;endTS;byCols]1+`}
                                                ^
q))table / See what current table argument is
`trade
q))startTS / See what current startTS argument is
-0Wp

Based on the output you can see specifically where the function is failing. Additionally, you can inspect the current values of the arguments. For larger functions, you can also run through the code line by line to identify the underlying issue.

Getting the backtrace of query or aggregations functions remotely

Set the environment variable KXI_DEBUG_QUERY to true on the resource-coordinators, or set debug to 1b in the opts of a request. When you do, if there is a failure in either a query function or an aggregation function, then the backtrace is returned in the header in the key bt.

Example with qipc:

q)GATEWAY:10i / Handle to gateway
q)args:enlist[`table]!enlist `trade
q)r:GATEWAY (`.kxi.qsql;enlist[`query]!enlist"1+`";`ignored;enlist[`debug]!enlist 1b) / The ignored is the callback function in async case
q)-1 r[0;`bt]
{}[0]   {1+`}
...

Testing aggregation functions

When creating a UDA, it is essential to develop an aggregation function that consolidates partial results from multiple DAPs. If a bug exists in the aggregation logic, identifying the root cause can be challenging when an aggregation fails.

To assist in debugging, there are options available to have the aggregator return the partial results of a failed aggregation back to the caller. This allows for debugging within the same session. When the partial results are returned, the response includes an application code (ac) in the header, describing the type of aggregation failure, and an rc value of 100h (PARTIALS).

If the partialsSent field is present and set to true, the payload consists solely of the partial results from each DAP involved in the request. By default, partial results are not sent when an aggregation fails. However, you can configure the aggregator to send back partial results using two methods:

Set the sendPartials value in the request header
Configure the aggregator to always send partial results

Step 1 - Enable partial results on aggregation failure

To debug failed aggregations, configure the aggregator to return partial results:

Option 1 - Set the sendPartials value in the request header
- Include the sendPartials field in the request header sent to the gateway. This approach ensures that the aggregator returns partial results if an aggregation failure occurs.
Example with qipc:
```
q)GATEWAY:10i / Handle to gateway
q)args:enlist[`table]!enlist `trade
q)GATEWAY (`.kxi.getData; args;`ignored;enlist[`sendPartials]!enlist 1b)
```
Option 2 - Configure the aggregator to always send partial results
- Set the environment variable KXI_SEND_PARTIALS to true on the aggregator. This configuration ensures that any failed aggregation request automatically sends partial results back in the payload.

Step 2 - Analyze the aggregator's response

If sendPartials is enabled and the request errors in the aggregator, the response contains the partial results. The response format depends on the specific error encountered:

Return Code = PARTIALS (100)

The payload is a list of partial responses from each DAP that participated in the request:

Example with qipc:

q)`rc`ac`ai#response 0
rc| 100h
ac| 30h
ai| "Unexpected error (mismatch) encountered aggregating myAPI"
q)response 1
+(,`x)!,1 2
+(,`y)!,3 4

Return Code = PARTIALS_SUB (101)
- If one or more sub-requests fail, the payload is a table that includes the response header and payload of each failed sub-request.
Return Code = PARTIALS_RESUME (102)
- The payload is a dictionary that contains partials, context, and callback information.

Timing your UDA functions (BETA)

Set the environment variable KXI_DEBUG_QUERY to true on the resource-coordinators, or set debug to 1b in the opts of a request to receive the times in milliseconds across various points of the query path.

Example with qipc:

q)GATEWAY:10i / Handle to gateway
q)args:enlist[`table]!enlist `trade
q)r:GATEWAY (`.uda.myUDA; args;`ignored;enlist[`debug]!enlist 1b) / The ignored is the callback function in async case
q)r[0;`timing] / Look at timing key of header
gw                  | 2
sg-rc-0             | 4
da-hdb-canada-0:5100| 10
da-rdb-canada-0:5080| 2
sg-agg-0-0          | 10

Explanation of timings:

gw key, measures the time from when the SG received the request, to the time the RC received the request.
rc key, display the hostname of the resource-coordinator that serviced the request and it measures time from the RC receiving the request to the request being issued to the DAP.
DAPs are marked by hostname and port (to disambiguate between multiple DAPs in a kxi-da-single image). The value measures time in milliseconds from getting the request to sending the partial result to the Aggregator.
agg timings will be in as its hostname and the value is the time it took to run the aggregation function.

If the request gets deferred or goes through a subquery, then the timing dictionary has one for each query or subquery in the routing path.

Example:

q)r[0;`timing]
                                      | ()!()
98a2a837-3154-4912-80eb-290ca6a128b7  | `gw`sg-rc-0`da-rdb-canada-0:5080`sg-agg-0-0!1 5 3 910
98a2a837-3154-4912-80eb-290ca6a128b7_1| `sg-rc-0`da-hdb-canada-0:5100`sg-agg-0-0!5 546 360