Code Examples and Templates

This page provides code examples and templates to accelerate your development process and ensure that your UDAs adhere to best practices.

Code Examples

1. Loading other file dependencies

// Custom UDA code.

.kxi.packages.file.load"common.q" / Example loading other file dependencies

2. UDA to provide the first 100 records from each DAP

Define the Query Function:

// DA query function
.example.queryFn:{[table;columns]
    columns:$[-11h = type columns;enlist columns;columns];
    filter:enlist (<;`i;100); // Note for partitioned tables, will return first 100 per date
    .kxi.response.ok ?[table;filter;0b;columns!columns]
    };

Define the Aggregation Function:

// Aggregation function
.example.aggFn:{[res]
    .kxi.response.ok 100?raze res
    };

Add metadata:

// Metadata for registering
metadata:.kxi.metaDescription["Returns first 100 records of each DAP"], / Description of the UDA
.kxi.metaParam[`name`type`isReq`description!(`table;-11h;1b;"Name of the table to query.")], / Describe `table` parameter
.kxi.metaParam[`name`type`isReq`description!(`columns;11 -11h;1b;"Columns in table to select")], / Describe `columns` parameter
.kxi.metaReturn`type`description!(98h;"Return first 100 rows."), / Describe the return of the UDA
.kxi.metaParam[`name`type`isReq`description!(`scope;-99h;0b;"Scope of the UDA to ensure the appropriate Aggregator is used")],
.kxi.metaMisc[enlist[`safe]!enlist 1b]; / States that this UDA is safe to be retried in the event of an error

Register the UDA:

// Registration
.kxi.registerUDA `name`query`aggregation`metadata!(`.example.testUDA`.example.queryFn;`.example.aggFn;metadata)

Once the UDA is added to a package and deployed the following requests can be made:
qREST
Gateway URL

The GATEWAY variable below is defined as an IPC connection to the Service Gateway. For example, `:insights-qe-gateway:5050 would connect to the query environment gateway within an insights namespace.
// args:`table`columns!("tablename;"column1,column2") // show each GATEWAY(`.example.testUDA;args;`;()!())
Gateway URL

The $GATEWAY variable should point at your kdb Insights install. For an SDK install, this will be the hostname of the install using port 8080. For an Enterprise install, this is your $INSIGHTS_HOSTNAME with /servicegateway as the URL prefix.
TABLE="tablename" COLUMNS="column1,column2" curl -X POST --header "Content-Type: application/json"\ --header "Accepted: application/json"\ --header "Authorization: Bearer $INSIGHTS_TOKEN"\ --data "{\"table\": \"$TABLE\", \"columns\": \"$COLUMNS\"}"\ "https://${INSIGHTS_HOSTNAME}/servicegateway/example/testUDA"

3. UDA to provide summary values

This UDA provides a summary of total and average usage by first 3 digits of postal codes in a table.

Example schema:

In this example, the table readings contains information about sensor usage and the Postal Code of the sensor.

q)readings
sensorID readTS                        val postal
-------------------------------------------------
0        2024.08.12D17:32:37.645609220 100 P0P1K0
1        2024.08.12D17:32:37.645609220 200 K7K2C7
q)meta readings
c       | t f a
--------| -----
sensorID| j
readTS  | p
val     | f
postal  | s

Define the Query Function:

The usageQuery function computes the count and sum of the val column, grouped by the first three digits of the postal code:

// @desc Computes the count of records as well as the sum of `val` column by first 3 digits of postal code.
//
// @param startTS   {timestamp}     Starting timestamp.
// @param endTS     {timestamp}     Ending timestamp.
// @param filter    {list}          Filter to apply to data.
//
// @return         {table}         Average count by postal code.
//
.example.usageQuery:{[startTS;endTS;filter]
    args:`table`startTS`endTS`columns`filter`groupBy!(`readings;startTS;endTS;`sumVal`cntVal!((sum;`val);(count;`val));filter;enlist[`postal]!enlist`postal); / Build arguments
    res:.kxi.selectTable args; / Make selectTable call
    res:0!select sumVal:sum sumVal,cntVal:sum cntVal by postal:`$3#/:string postal from res; / Anonymize data, and perform further processing
    .kxi.response.ok res / Return with OK
    };

Define the Aggregation Function:

The usageAgg function consolidates partial results from DAPs and calculates the total usage statistics, including count and average per postal code:

// @desc Aggregates partial results from DAPs to produce total usage statistics, count of records, and average per postal code.
//
// @param tbls  {table[]}   List of tables with postal, sumVal, and cntVal columns.
//
// @return      {table}     Table with sum, count, and average by first 3 digits of postal code.
//
.example.usageAgg:{[tbls]
    .kxi.response.ok 0!update avgVal:sumVal%cntVal from select sumVal:sum sumVal, cntVal:sum cntVal by postal from raze tbls
    };

Add metadata:

Define metadata for the UDA to describe its purpose, parameters, and return values:

// Define metadata. 
metadata:.kxi.metaDescription["Returns the average by postal code across the first 100 records of each DAP"], / Description of the UDA
    .kxi.metaParam[`name`type`isReq`description!(`startTS;-12h;1b;"Start timestamp of query")], / Describe `startTS` parameter
    .kxi.metaParam[`name`type`isReq`description!(`endTS;-12h;1b;"End timestamp of query")], / Describe `endTS` parameter
    .kxi.metaParam[`name`type`isReq`default`description!(`filter;0h;0b;();"Filter to apply to readings table")], / Describe `filter` parameter
    .kxi.metaReturn`type`description!(98h;"The summary of total and average usage by first 3 digits of postal code"), / Describe the return of the UDA

Register the UDA:

Register the UDA with the appropriate names for query and aggregation functions, and provide the metadata:

.kxi.registerUDA `name`query`aggregation`metadata!(`.example.usageByPostalCode;`.example.usageQuery;`.example.usageAgg;metadata) / Register UDA

Once the UDA is added to a package and deployed the following requests can be made:
qREST
Gateway URL

The GATEWAY variable below is defined as an IPC connection to the Service Gateway. For example, `:insights-qe-gateway:5050 would connect to the query environment gateway within an insights namespace.
// args:`startTS`endTS!("p"$.z.D-3;"p".z.D-2) // show each GATEWAY(`.example.usageByPostalCode;args;`;()!())
Gateway URL

The $GATEWAY variable should point at your kdb Insights install. For an SDK install, this will be the hostname of the install using port 8080. For an Enterprise install, this is your $INSIGHTS_HOSTNAME with /servicegateway as the URL prefix.
startTS=$(date -u '+%Y.%m.%dD%H:00:00') endTS=$(date -u '+%Y.%m.%dD%H:%M%:%S') curl -X POST --header "Content-Type: application/json"\ --header "Accepted: application/json"\ --header "Authorization: Bearer $INSIGHTS_TOKEN"\ --data "{\"startTS\": \"$startTS\", \"endTS\": \"$endTS\"}"\ "https://${INSIGHTS_HOSTNAME}/servicegateway/example/usageByPostalCode"

4. UDA to join Trade and Quote tables

UDA definition:

\d .customUDA

// Join Trade and Quote tables

// Define query function.
//
// @desc Define a new API. Get trade data for specified date range.
// @param startTS   {timestamp}         Start time (inclusive).
// @param endTS     {timestamp}         End time (exclusive).
// @param s         {symbol|symbol[]}   Market data symbols to query for.
// @return          {table}             Count by specified columns.
queryMktData:{[startTS;endTS;syms]
    if[`~syms;
        :.kxi.response.error[.kxi.response.ac.INPUT;"Must enter a non null symbol(s) to query";()]];
    symFilt:(in;`sym;enlist syms);
    filter:((within;`realTime;(startTS;endTS));symFilt);
    t:.kxi.selectTable`table`startTS`endTS`filter!(`trade;startTS;endTS;filter);
    .kxi.response.ok t
    }

// Define aggregation function.
//
// @desc Get symbol and timerange for quote data, then defer the final result until we retrieve the quote data.
// @param trade {table}     Trade data from the query result.
aggMktData:{[trade]
    trade:raze trade; / Raze as data coming from multiple DAPs
    .kxi.context.set[`trade;trade]; / Save trade data in context
    symFilt:exec distinct sym from trade;
    args:`table`startTS`endTS`agg`sortCols!
        (`quote;"p"$"d"$min trade`realTime;1+max trade`realTime;
        `sym`realTime`ask`askSize`bid`bidSize;`sym`realTime); / Args for quote getData
    .kxi.response.callAPI[`.kxi.getData;args;`.customUDA.aggMktDataResume;()!()]
    }

// Aggregation defer resume.
// @desc    Resume deferred request and join together trade and quote data.
// @param   quote   {table}     Quote data.
// @return  Joined trade and quote data for specified syms with numeric values rounded to 2 decimal places.
aggMktDataResume:{[quote]
    trade:.kxi.context.get`trade; / Recover trade data from context
    res:aj[`sym`realTime;trade;quote]; / Join
    round:{("j"$100*x)%100}; / Round to two decimals
    .kxi.response.ok update round price,round bid,round ask from res
    }

\d .

// Define metadata.
metadata:.kxi.metaDescription["Custom UDA - join trade and quote data."],
    .kxi.metaMisc[enlist[`safe]!enlist 1b],
    .kxi.metaParam[`name`type`isReq`description!(`startTS;-12h;1b;"Start time (inclusive).")],
    .kxi.metaParam[`name`type`isReq`description!(`endTS;-12h;1b;"End time (exclusive).")],
    .kxi.metaParam[`name`type`isReq`description!(`syms;11 -11h;1b;"Sym values to query on. Must be a non-null value.")],
    .kxi.metaReturn`type`description!(98h;"Joined trade and quote data.");

// Registration.
.kxi.registerUDA `name`query`aggregation`metadata!(`.customUDA.getMktData;`.customUDA.queryMktData;`.customUDA.aggMktData;metadata);

Once the UDA is added to a package and deployed the following requests can be made:

qREST

Gateway URL

The GATEWAY variable below is defined as an IPC connection to the Service Gateway. For example, `:insights-qe-gateway:5050 would connect to the query environment gateway within an insights namespace.

// Example qIPC requests.
// GATEWAY variable below is defined as an IPC connection to the Service Gateway.
// args:`startTS`endTS`syms!("p"$.z.D-3;"p".z.D-2;`DNDF.CAN)
// show each GATEWAY(`.customUDA.getMktData;args;`;()!())

Gateway URL

The $GATEWAY variable should point at your kdb Insights install. For an SDK install, this will be the hostname of the install using port 8080. For an Enterprise install, this is your $INSIGHTS_HOSTNAME with /servicegateway as the URL prefix.

startTS=$(date -u '+%Y.%m.%dD%H:00:00')
endTS=$(date -u '+%Y.%m.%dD%H:%M%:%S')
syms="syms"
curl -X POST --header "Content-Type: application/json"\
    --header "Accepted: application/json"\
    --header "Authorization: Bearer $INSIGHTS_TOKEN"\
    --data "{\"startTS\": \"$startTS\", \"endTS\": \"$endTS\",\"syms\": \"$syms\"}"\
    "https://${INSIGHTS_HOSTNAME}/servicegateway/customUDA/getMktData"