Advanced UDA configuration

This page details advanced UDA configuration options for kdb Insights.

Loading UDAs into DAPs and Aggregators

There are two ways UDAs can be deployed:

As part of the database package: UDAs added to a database package can only be used by that database. The Resource Coordinator routes requests only to DAPs and Aggregators that have the UDA registered.
In a code-only package: This assists with code separation and allows UDAs to be loaded into DAPs and Aggregators in more than one database.

Loading code-only packages

Loading a code-only package into running DAPs and Aggregators using kxi pm load, applies the changes only for the lifetime of those processes. When the processes are restarted, the updates are lost. To ensure the changes are permanent and the DAPs and Aggregators load the UDAs on startup, the database package must include a reference to the code-only package in the KXI_PACKAGES environment variable.

Loading code-only packages into database processes

To ensure the UDAs are loaded into the DAPs and dedicated Aggregators of a database on deployment the package needs to include deployment configuration. Refer to the dedicated Aggregator for details on why you would use a dedicated Aggregator.

Add a deployment configuration file to the database package using kxi package add:

kxi package add --to db-package deployment-config

This creates a deployment_config.yaml file in a deployment_config folder of the package as follows:

└──uda-package
    ├── databases
    │   └── uda-package
    │       ├── shards
    │       │   └── uda-package-shard.yaml
    │       └── tables
    ├──deployment_config
    │   └── deployment_config.yaml
    ├── init.q
    ├── manifest.yaml
    └── src
        └── uda.q

Add the KXI_PACKAGES environment variable to this file, setting the code-only package as the value:
```
env:
- name: KXI_PACKAGES
  value: "uda-package"
```
Note
- If you do not specify a version the latest will always be loaded.
- If you wish to load a specific version of a package you can append the version number to the package name as follows:
```
env:
- name: KXI_PACKAGES
  value: "uda-package-1.0.0"
```
Note

If you need to load multiple packages you can use a comma separated list.
```
env:
- name: KXI_PACKAGES
  value: "uda-package:1.0.0,another-pkg"
```

Note

When editing a package, we recommend updating the version number to show that it has been updated. The example below increases the minor version:

kxi package checkpoint db-package --bump minor

To deploy these changes the database and code-only packages need to be pushed to kdb Insights Enterprise and the database package needs to be redeployed to ensure the DAPs and Aggregators load the UDA on startup.

Loading UDAs into global Aggregators

When a package does not include dedicated aggregators the global aggregator executes the aggregation functions. Refer to the Aggregator details for reasons why you would use a global aggregator.

There are different methods of loading UDAs into the global aggregator.

During testing, if your package is deployed, running kxi pm load dynamically updates the aggregator.

Warning

These changes are lost if the aggregator is restarted or the system is upgraded.
If you have cluster access, you can update the aggregators by editing the stateful set. These changes persist across Aggregator restarts, but are lost during a system upgrade.

Warning

These changes are lost if the aggregator the system is upgraded.
- If Query Environments are enabled, update the QE Aggregator:
```
kubectl edit sts insights-qe-aggregator
```
  Add KXI_PACKAGES to the list of environment variables in spec.template.spec.containers.env:
```
- name: KXI_PACKAGES
  value: uda-package:1.0.0,another-pkg
```
  Note
  
  During the development and testing of the UDA, if Query Environments are enabled, we recommend you test the UDA on the query environment first and only update the Aggregator on the main query path once you are confident the UDA is working as expected.
- If you don't have Query Environments enabled or you are ready to deploy to the main query path, you must update the main Aggregator:
```
kubectl edit sts insights-aggregator
```
  Add KXI_PACKAGES to the list of environment variables in spec.template.spec.containers.env:
```
- name: KXI_PACKAGES
  value: uda-package:1.0.0,another-pkg
```
To ensure the KXI_PACKAGES environment variable is retained after an upgrade, it needs to be added to the values.yaml file. Contact your administrator, and refer to Applying configuration changes for details on how to add to the values.yaml file.
- If Query Environments are enabled, update the query environment Gateway settings as follows:
```
qe-service-gateway:
aggregator:
    env:
    - name: KXI_PACKAGES:
        value: uda-package:1.0.0,another-pkg
```
- Update the main query gateway settings:
```
service-gateway:
aggregator:
    env:
    - name: KXI_PACKAGES:
        value: uda-package:1.0.0,another-pkg
```

Adding logging to UDAs

You can use the logging wrapper functions provided by kdb Insights, which streamlines the logging process provided by Qlog or.

Make sure logging is enabled.

When debugging UDA issues, you can also follow the guidance in the UDA testing documentation and inspect DAP and Aggregator logs alongside your own log entries.
Initialize logging in each code file:
```
\d .example
.log.initns[]
```
Add log entries to the query and aggregation functions, making sure to use the namespace in the function call as follows:
```
.example.log.info"Hello, world"
```

Recommendations

To assist with debugging debugging to have the following log entries as a minimum:

A debug log entry that takes the args dictionary or parameters
A completion log entry
Additional log entries specific to your code

Example:

Using argsUsing multiple parameters

\d .tca

.log.initns[];

.uda.example: {[args]
    / Log start with parameters
    log.debug("Starting .uda.example args=%d";args);

    //UDA code

    / Log that the function is complete
    .uda.log.info ".uda.example: complete";

    }

\d .uda

.log.initns[];

.uda.example:{[param1;param2]
    / Log start with parameters
    .uda.log.debug enlist[`param1`param2!
            ("Starting .uda.example"; param1; param2)];

    //UDA code

    / Log that the function is complete
    .uda.log.info ".uda.example: complete";

    }