Installing User Defined Analytics

User Defined Analytics (UDAs) allow you to define new APIs for querying and aggregating data in databases.

Note

Users who need to call UDAs must have the insights.query.custom permission, which is included in the Viewer Composite role.

Steps to install a UDA

This section explains how to deploy a UDA in kdb Insights Enterprise that counts the number of records for specified columns and tables within a selected time range, as defined in the Creating a UDA and How to add a UDA to a package documentation.

Prerequisites

Before deploying a package containing a UDA, ensure the following prerequisites are met:

Access to a kdb Insights instance
The kxi CLI is installed and configured on your system.

Preparing the UDA

Prepare the UDA package by following the UDA creation guide.
Add the UDA to a package by following the How to add a UDA to a package guide.

Note

If you are unfamiliar with how packaging works in kdb Insights Enterprise, it is recommended to review this topic before continuing. This foundational knowledge is essential for understanding how UDAs are packaged and deployed. Refer to the Package Lifecycle documentation and the packaging Quickstart for more information.

Deploying the UDA

Ensure the UDA is loaded when the database is deployed
1. Load the UDA into the database processes that are part of the package:
  
  Add environment flags to the package containing the database definition. This configuration ensures the UDAs are loaded into processes with defined entrypoints when the package is deployed. For example:
```
kxi package add --to mypackage deployment-config
echo "env: [{name: KXI_PACKAGES, value: mypackage}]" >> mypackage/deployment_config/deployment_config.yaml
```
2. If the package is to be configured to use a dedicated Aggregator and Resource Coordinator as part of your database package, the aggregation function must be loaded into this dedicated Aggregator when the package is deployed.
  1. Add an Aggregator and Resource Coordinator to the package:
```
kxi package add --to mypackage agg
kxi package add --to mypackage rc
```
  2. Update the metadata to include a scope variable. This variable ensures that the dedicated Aggregator is used when calling the UDA.
    
    Add a scope parameter to the metaDescription function in your UDA. The example in the creating UDAs section is updated as follows:
```
// Define metadata.
metadata:.kxi.metaDescription["Custom UDA - does a count by."],
    .kxi.metaMisc[enlist[`safe]!enlist 1b],
    .kxi.metaParam[`name`type`isReq`description!(`table;-11h;1b;"Table name.")],
    .kxi.metaParam[`name`type`isReq`description!(`byCols;11 -11h;1b;"Column(s) to count by.")],
    .kxi.metaParam[`name`type`isReq`description!(`startTS;-12h;1b;"Start time (inclusive).")],
    .kxi.metaParam[`name`type`isReq`description!(`endTS;-12h;1b;"End time (exclusive).")],
    .kxi.metaParam[`name`type`isReq`description!(`scope;-99h;0b;"Scope of the UDA to ensure the appropriate Aggregator is used")],
    .kxi.metaReturn`type`description!(98h;"Count by specified columns.");
```
  Info
  
  Ensure that you populate the scope value when calling the UDA to confirm that the dedicated components are used. Refer to the Querying UDAs documentation for more details
Push and deploy the package to kdb Insights Enterprise

Note

If your UDA is not in the package containing the database definition, complete the following steps for both packages.
1. Refer to the Authentication documentation for authentication details.
2. Use the KXI CLI to create users, and assign roles, if they are not already present. For more information on the CLI, refer to the kdb Insights Command Line documentation.
  
  Administrator password
  
  The administrator password below is defined during the installation of your kdb Insights Enterprise deployment. For more information, refer to the Administration Passwords documentation.
  
  The example below adds a user and gives them the insights.role.maintainer role:
```
# Create a user
kxi user create user@domain.com --password <myPassword> --admin-password <adminPassword> --not-temporary
INSIGHTS_ROLES="insights.role.maintainer"
kxi user assign-roles user@domain.com --roles $INSIGHTS_ROLES --admin-password <adminPassword> 
```
3. Use the kxi pm push command to upload your package:
```
kxi pm push mypackage
```
  When configured correctly, kdb Insights Enterprise displays a confirmation that the push command succeeded.
  
  Note
  
  If you have defined your UDA as part of a code-only package you must push both the database package and the code-only package to kdb Insights Enterprise.
4. Use the kxi pm deploy command to deploy your package:
```
kxi pm deploy mypackage
```
  Note
  
  You only need to deploy the database package. The environment variables added to the database package ensure any UDAs in the code-only package are loaded.
5. If you are using the global Aggregator included in the kdb Insights Enterprise base installation, rather than a dedicated Aggregator defined in the package, you must also load the aggregation function into the global Aggregator.
  
  Note
  
  All packages created through the UI use the global Aggregator.
  
  The following steps requires cluster access to edit the stateful set. Add the KXI_PACKAGES environment variable to the list of environment variables in spec.template.spec.containers.env for the global Aggregator components:
  1. Update the Aggregator:
```
kubectl edit sts insights-aggregator
```
    Add KXI_PACKAGES to the list of environment variables in spec.template.spec.containers.env:
```
- name: KXI_PACKAGES
value: mypackage
```
  2. Update the QE Aggregator, used for exploratory queries:
```
kubectl edit sts insights-qe-aggregator
```
    Add KXI_PACKAGES to the list of environment variables in spec.template.spec.containers.env:
```
- name: KXI_PACKAGES
value: mypackage
```
Test the UDA

To test the UDA, you can call the UDA in the following ways:
- Query window
- Views
- REST

Handling Unsuccessful Deployments

Identify Errors

If the deployment fails, review error messages and logs to identify the cause. Common issues include missing dependencies, incorrect configurations, or permission errors.

Roll Back Changes

If necessary, roll back the deployment to the last known stable state to avoid disruptions in production.

Troubleshoot and Redeploy

Identify and troubleshoot issues, correct any errors, and attempt the deployment again.