Skip to content

Load Multiple Packages into a DAP

This walkthrough shows how to load code from several packages into a single Data Access Process (DAP) using package dependencies.

A DAP runs a database tier of a package and serves queries and User-Defined Analytics (UDAs). It is common to want a DAP to also run code that is maintained in other packages — for example, a shared utility module, or a library of analytics reused across databases. The recommended way to do this is to declare those packages as dependencies of the package that owns the database; they are then resolved and loaded automatically on deploy.

For the format and resolution mechanics, refer to Package Dependency. This page focuses on the end-to-end steps.

In this walkthrough you will:

  1. Create and push a reusable utility module (myutils).
  2. Create a database package (trades-db) that depends on myutils.
  3. Load the module inside a UDA that runs on the DAP.
  4. Push and deploy the package, and verify both packages are loaded into the DAP.

Prerequisites

  • The kxi CLI installed and authenticated against your kdb Insights Enterprise environment.
  • The insights.admin.package.system role, which is part of the Administrator role, required to push and deploy packages.

1. Create the utility module

A module is a reusable q library with an isolated namespace that exposes a public API through an export dictionary. Create one and add an entrypoint:

kxi package init myutils --force

Define the module's public API in myutils/init.q:

.myutils.greet:{[name] raze "Hello, ",name}
.myutils.PI:3.14159

export:([greet:.myutils.greet; PI:.myutils.PI])

Set the module name and version in myutils/manifest.yaml:

name: myutils
version: 1.0.0

Push the module to kdb Insights Enterprise so it is available system-wide for any package to depend on:

kxi pm push myutils

2. Create the database package

Create the package that owns the database and the UDA:

kxi package init trades-db --force
kxi package add --to trades-db database --name trades-db
kxi package add --to trades-db table --name trades

Declare the dependency

Add myutils as a dependency. A version specifier is optional but recommended:

kxi package add --to trades-db dep --name myutils --version 1.0.0

This adds the dependency to trades-db/manifest.yaml in the compact string form:

dependencies:
  - myutils==1.0.0

To depend on more than one package, add a dep for each. They are all resolved and loaded into the DAP:

kxi package add --to trades-db dep --name analytics --version 2.0.0

dependencies:
  - myutils==1.0.0
  - analytics==2.0.0

3. Load the modules in a UDA

Code that runs on the DAP is registered in the data-access entrypoint. Add the entrypoint file:

kxi package add --to trades-db entrypoint --name data-access --path src/da.q

In trades-db/src/da.q, load each dependency with use and call its exported API inside the UDA query function that runs on each DAP:

.utils: use `myutils;

.myuda.query:{[table;startTS;endTS]
    res:.kxi.selectTable`table`startTS`endTS!(table;startTS;endTS);
    res:update label:.utils.greet each string sym from res;
    .kxi.response.ok res
    };

metadata:.kxi.metaDescription["Greet each sym in the trades table."],
    .kxi.metaParam[`name`type`isReq`description!(`table;-11h;1b;"Table name.")],
    .kxi.metaParam[`name`type`isReq`description!(`startTS;-12h;1b;"Start time.")],
    .kxi.metaParam[`name`type`isReq`description`default!(`endTS;-12h;0b;"End time.";.z.P)];

.kxi.registerUDA`name`query`metadata!(`.myuda;`.myuda.query;metadata);

The resulting manifest.yaml ties everything together — the dependency, the entrypoint that loads it, and the database:

name: trades-db
version: 1.0.0
dependencies:
  - myutils==1.0.0
entrypoints:
  data-access: src/da.q
tables:
  trades:
    file: tables/trades.yaml
databases:
  trades-db:
    dir: databases/trades-db
    shards:
      - trades-db

Use use, not \l

Inside a package, load dependencies with use (or .Q.m.reuse, see Hot-reload). Do not use \l — it bypasses module resolution.

4. Push and deploy

Push the package and deploy it. Before the DAP starts, myutils and any other declared dependency are resolved:

kxi pm push trades-db
kxi pm deploy trades-db

5. Verify the packages are loaded

Confirm the deployment is running:

kxi pm list

Behind the scenes, the resolved set is loaded into each DAP — trades-db plus every declared dependency, with the entrypoint files from each loaded in order (trades-db/src/da.q and myutils/init.q in this example).

You can now call the UDA from the Query Window, Views or REST; the label column it returns is produced by the myutils module running inside the DAP.

Chaining dependencies

Dependencies are resolved transitively: if analytics itself depends on myutils, declaring only analytics pulls in myutils too. Push each layer before the package that depends on it. See transitive dependencies for details.

Hot-reload packages in a running DAP

During development you can push a new version of a dependency into running DAPs and Aggregators without a full redeployment, using kxi pm load:

kxi pm load myutils --version 1.1.0 -d trades-db -t daps,pkg_agg

For the reload to take effect, the module must be loaded with .Q.m.reuse instead of plain use. In trades-db/src/da.q, change the load line from step 3 to:

.utils: .Q.m.reuse `myutils;

.Q.m.reuse loads the module on first call and reloads it on subsequent calls, so a hot-reload picks up the new version. Plain use caches the first-loaded version, so a kxi pm load would have no effect — use it when you do not need hot-reload, and .Q.m.reuse during development when you do.

Warning

kxi pm load only updates the running process. It does not change the package's saved dependencies, so the update does not survive a restart or redeployment. To make it permanent, bump the dependency version in manifest.yaml and redeploy.

Global Aggregator

The Global Aggregator (global_agg) included in the base installation is not managed by the Package Manager, so it is the one place where declared dependencies are not resolved automatically.

Wherever possible, ship a dedicated agg component in your package so its aggregation code is resolved and loaded from dependencies like any other component. If a UDA's aggregation function must run on the Global Aggregator, load the package into it at runtime with kxi pm load:

kxi pm load myutils --version 1.0.0 -d trades-db -g agg

See Use code in a package for more on the global Aggregator, and Advanced UDAs for loading UDAs into Global Aggregators.

Next steps

Back to top