Skip to content

Automated Package Deployment

This page outlines how packages can be deployed automatically with CI/CD processing.

The recommended approach is to use the CLI for automated deployments.

However, as kdb Insights Enterprise is OpenAPI-compliant with support API endpoints for all package operations, the same operations can be performed through API calls by any REST-compliant client.

KXI CLI

The KXI CLI is the primary method to deploy packages.

This is a command line tool built on Python which offers zero prompt interactions with packages in all APIs.

Read Install KXI CLI for information on installing the CLI.

CLI prerequisites

Object Format
Insights URL $INSIGHTS_URL
client-id $INSIGHTS_CLIENT_ID
client-secret $INSIGHTS_CLIENT_SECRET
Realm (optional) $INSIGHTS_REALM
Package file $PACKAGE_NAME-$PACKAGE_VERSION.kxi
Package location for example KX portal

Ensure the following:

  • You are running kdb Insights Enterprise version 1.12 or later

  • You have a valid service account with the necessary roles and entitlements

  • The package is published and accessible from a REST client

  • curl is available on the machine executing the deployment

  • KXI CLI v1.12+ is installed on the deployment machine. Make sure the CLI version matches the kdb Insights Enterprise version

  • The CLI is configured with the required credentials and hostname.

Steps using CLI

This example uses a package called kxi-db on package version 1.14.0.

set -euo pipefail
source .env

PACKAGE_NAME="kxi-db"
PACKAGE_VERSION="1.14.0"
PACKAGE_FILE="${PACKAGE_NAME}-${PACKAGE_VERSION}.kxi"

curl --silent --location --remote-name \
  --fail-with-body \
  --oauth2-bearer "$BEARER" \
  "https://portal.dl.kx.com/assets/raw/packagesamples/${PACKAGE_NAME}/${PACKAGE_VERSION}/${PACKAGE_FILE}"

# kxi pm reads INSIGHTS_URL, INSIGHTS_CLIENT_ID, INSIGHTS_CLIENT_SECRET,
# and optionally INSIGHTS_REALM from the environment.
kxi pm push "$PACKAGE_FILE" --force
kxi pm deploy "$PACKAGE_NAME" "$PACKAGE_VERSION"
kxi pm list deployment \
  --filter "name=${PACKAGE_NAME}" \
  --output-format json
You can use the help command to provide information regarding the additional options. Expand for details 

 Usage: kxi pm deploy [OPTIONS] PACKAGE [VERSION]                               

 Deploy a package to an insights instance.                                      
 PACKAGE: package-name VERSION: package-version                                 
 With --template, deploy the named template (.j2 stem) against PACKAGE as a     
 standalone instance; --name sets the instance name and --set/-f supply the     
 template values.                                                               

╭─ Authentication option overrides ────────────────────────────────────────────╮
│ --hostname,--url                TEXT  Insights URL                           │
│                                       [env: INSIGHTS_URL]                    │
│ --realm                         TEXT  Realm                                  │
│                                       [env: INSIGHTS_REALM]                  │
│ --client-id                     TEXT  Client id                              │
│                                       [env: INSIGHTS_CLIENT_ID]              │
│ --client-secret                 TEXT  Client secret                          │
│                                       [env: INSIGHTS_CLIENT_SECRET]          │
│ --auth-enabled/--auth-disabled        Retrieve Bearer Token                  │
│                                       [env: KXI_AUTH_ENABLED]                │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --env                   TEXT          Inject environment variables to the    │
│                                       deployed package.                      │
│                                       `[component_name:]VAR=value`           │
│ --db                    TEXT          Deploy a database in the package       │
│ --pipeline              TEXT          Deploy a pipeline in the package       │
│ --template              TEXT          Deploy the named template (.j2 stem)   │
│                                       against the package as an instance.    │
│ --name                  TEXT          Name for the resulting deployment      │
│                                       (defaults to the package / template    │
│                                       manifest).                             │
│ --values            -f  FILE          Load template values from a YAML file. │
│                                       Multiple files are merged              │
│                                       left-to-right.                         │
│ --set-file              TEXT          Set a template value from a file's     │
│                                       contents. KEY=FILEPATH                 │
│ --set                   TEXT          Override a template value at push      │
│                                       time. KEY=VALUE                        │
│ --fields                TEXT          [default: name, package, version,      │
│                                       function, status, data, access, owner, │
│                                       id, info, size, components]            │
│ --force                               Overwrite without prompting for        │
│                                       confirmation.                          │
│ --output-format     -o  [json|table]  Output format for the command.         │
│                                       default: 'table'.                      │
│ --server-timeout        INTEGER       Timeout for Insights server calls      │
│                                       [env: INSIGHTS_SERVER_TIMEOUT]         │
│ --yes,--assume-yes  -y                Automatic yes to prompts; assume "yes" │
│                                       as answer to all prompts and run       │
│                                       non-interactively.                     │
│ --help                                Show this message and exit.            │
╰──────────────────────────────────────────────────────────────────────────────╯

curl

The OpenAPI endpoints can be used for all operations, allowing any REST-compliant client to perform the same operations as KXI CLI.

curl prerequisites

Object Format
client-id $INSIGHTS_CLIENT_ID
client-secret $INSIGHTS_CLIENT_SECRET
Package file $PACKAGE_NAME-$PACKAGE_VERSION.kxi
Package location For example KX portal

Ensure the following:

  • You are running kdb Insights Enterprise version 1.12 or later
  • You have created a client capable of authenticating to Insights
  • You have created a client that has the necessary roles and entitlements to deploy packages
  • You have published the package and made it available in a repository accessible from a REST client
  • You have curl available on the machine performing the deployment

Steps using curl

  1. Create a service account to permit authentication.

  2. Specify and save your client-id and client-secret.

  3. Add authorizations to permit pushing or deploying packages: assign "insights.role.maintainer" role.

  4. Locate the required package and make it available to a pipeline in a repository accessible from a REST client.

  5. Create curl script to:

    • Pull package from repository; for example Git, Nexus, portal.dl.kx.com etc
    • Push and deploy package to kdb Insights Enterprise
    • Retrieve status of package
set -euo pipefail

# Pull in secrets. This could be from a secure secrets store, or an .env file
# only accessible to the automated system. Provides BEARER (package
# repository), INSIGHTS_CLIENT_ID, and INSIGHTS_CLIENT_SECRET.
source .env

# Package details
PACKAGE_NAME="kxi-db"
PACKAGE_VERSION="1.14.0"
PACKAGE_FILE="${PACKAGE_NAME}-${PACKAGE_VERSION}.kxi"

# kdb Insights URL - update to an accessible endpoint
INSIGHTS_URL="https://insights.domain.com"

# Pull the package from the package repository
curl --silent --location --remote-name \
  --fail-with-body \
  --oauth2-bearer "$BEARER" \
  "https://portal.dl.kx.com/assets/raw/packagesamples/${PACKAGE_NAME}/${PACKAGE_VERSION}/${PACKAGE_FILE}"

# Retrieve an INSIGHTS_BEARER token. This is short-lived and must be renewed.
INSIGHTS_BEARER=$(curl --silent --fail-with-body --request POST \
  --header 'Content-Type: application/x-www-form-urlencoded' \
  --data-urlencode "client_id=${INSIGHTS_CLIENT_ID}" \
  --data-urlencode "client_secret=${INSIGHTS_CLIENT_SECRET}" \
  --data-urlencode 'grant_type=client_credentials' \
  "${INSIGHTS_URL}/realms/${INSIGHTS_REALM:-insights}/protocol/openid-connect/token" \
  | jq -r '.access_token')

# Push the package to kdb Insights Enterprise
curl --fail-with-body --request POST \
  --header "Authorization: Bearer ${INSIGHTS_BEARER}" \
  --form "data=@${PACKAGE_FILE}" \
  "${INSIGHTS_URL}/packagemanager/v2/artifacts?force=true"

# Deploy the package
curl --fail-with-body --request POST \
  --header "Authorization: Bearer ${INSIGHTS_BEARER}" \
  --header 'Content-Type: application/json' \
  --data "{\"data\":{\"type\":\"deployment\",\"attributes\":{\"package\":\"${PACKAGE_NAME}\",\"version\":\"${PACKAGE_VERSION}\"}}}" \
  "${INSIGHTS_URL}/packagemanager/v2/deployments"

To retrieve the status of the deployment, use the CLI command shown in the Steps using CLI section:

kxi pm list deployment \
  --filter "name=${PACKAGE_NAME}" \
  --output-format json

Next steps

Back to top