Skip to content

Automated Package Deployment

This page outlines how packages can be deployed automatically with CI/CD processing.

The recommended approach is to use the CLI for automated deployments.

However, as kdb Insights Enterprise is OpenAPI-compliant with support API endpoints for all package operations, the same operations can be performed through API calls by any REST-compliant client.

KXI CLI

The KXI CLI is the primary means to deploy packages.

This is a command line tool built on Python which offers zero prompt interactions with packages in all APIs.

Read Install KXI CLI for information on installing the CLI.

CLI prerequisites

Object Format
client-id $INSIGHTS_CLIENT_ID
client-secret $INSIGHTS_CLIENT_SECRET
Package file $PACKAGE_NAME-$PACKAGE_VERSION.kxi
Package location for example KX portal

Ensure the following:

  • You are running kdb Insights Enterprise 1.12+

  • You have created a client capable of authenticating to Insights

  • You have created a client that has the necessary roles and entitlements to deploy packages

  • You have published the package and made it available in a repository accessible from a REST client

  • You have curl available on the machine performing the deployment

  • KXI CLI v1.12 or later installed and available on the machine performing the deployment

  • KXI CLI configured to the Insights deploy

Steps using CLI

This example uses a package called kxi-db on package version 1.12.0.

# Pull in secrets. This could be from a secure secrets store or an .env
# only accessible to the automated system BEARER (package repository),
# INSIGHTS_CLIENT_ID (insights client name), INSIGHTS_CLIENT_SECRET source .env

# Package Details
PACKAGE_NAME="kxi-db"
PACKAGE_VERSION="1.0.0"

# Pull the package from the package repository
# BEARER provided from .env

curl -s -L -OJ --fail-with-body -D /dev/stderr --oauth2-bearer $BEARER
https://portal.dl.kx.com/assets/raw/packagesamples/$PACKAGE_NAME/$PACKAGE_VERSION/$PACKAGE_NAME-$PACKAGE_VERSION.kxi

# Push the package to the Insights deploy 
# INSIGHTS_CLIENT_ID, INSIGHTS_CLIENT_SECRET provided from .env

kxi pm push --client-id=$INSIGHTS_CLIENT_ID --client-secret=$INSIGHTS_CLIENT_SECRET --force
${PACKAGE_NAME}-${PACKAGE_VERSION}.kxi

# Deploy the package
kxi pm deploy --client-id=$INSIGHTS_CLIENT_ID --client-secret=$INSIGHTS_CLIENT_SECRET ${PACKAGE_NAME}

# Retrieve status
kxi pm list --client-id=$INSIGHTS_CLIENT_ID --client-secret=$INSIGHTS_CLIENT_SECRET ${PACKAGE_NAME} -o json 
You can use the help command to provide information regarding the additional options

 Usage: kxi pm deploy [OPTIONS] PACKAGE [VERSION]                               

 Deploy a package to an insights instance.                                      
 PACKAGE: package-name VERSION: package-version                                 

╭─ Authentication option overrides ────────────────────────────────────────────╮
│ --hostname,--url                  TEXT  Insights URL                         │
│ --realm                           TEXT  Realm                                │
│ --client-id                       TEXT  Client id                            │
│ --client-secret                   TEXT  Client secret                        │
│ --auth-enabled/--auth-disabled          Retrieve Bearer Token                │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --env                 TEXT          Inject environment variables to the      │
│                                     deployed package.                        │
│                                     `[component_name:]VAR=value`             │
│ --output-format   -o  [json|table]  Output format for the list command       │
│ --server-timeout      TEXT          Timeout for Insights server calls        │
│ --help                              Show this message and exit.              │
╰──────────────────────────────────────────────────────────────────────────────╯

curl

The OpenAPI endpoints can be used for all operations, allowing any REST-compliant client to perform the same operations as KXI CLI.

curl prerequisites

Object Format
client-id $INSIGHTS_CLIENT_ID
client-secret $INSIGHTS_CLIENT_SECRET
Package file $PACKAGE_NAME-$PACKAGE_VERSION.kxi
Package location For example KX portal

Ensure the following:

  • You are running kdb Insights Enterprise 1.12+
  • You have created a client capable of authenticating to Insights
  • You have created a client that has the necessary roles and entitlements to deploy packages
  • You have published the package and made it available in a repository accessible from a REST client
  • You have curl available on the machine performing the deployment

Steps using curl

  1. Create a service account to permit authentication.

  2. Specify and save your client-id and client-secret.

  3. Add authorizations to permit pushing or deploying packages: assign "insights.role.maintainer" role.

  4. Locate the required package and make it available to a pipeline in a repository accessible from a REST client.

  5. Create curl script to:

    • Pull package from repository; for example Git, Nexus, portal.dl.kx.com etc
    • Push and deploy package to kdb Insights Enterprise
    • Retrieve status of package
# Pull in secrets. This could be from a secure secrets store or an .env
# only accessible to the automated system BEARER (package repository),
INSIGHTS_CLIENT_ID (insights client name), INSIGHTS_CLIENT_SECRET
source .env

# Package Details
PACKAGE_NAME="kxi-db"
PACKAGE_VERSION="1.0.0"

# kdb Insights URL - please update to accessible endpoint.
#Example:
INSIGHTS_URL="https://insights.domain.com"

# Pull the package from the package repository

# BEARER provided from .env
curl -s -L -OJ --fail-with-body \-D /dev/stderr --oauth2-bearer $BEARER
https://portal.dl.kx.com/assets/raw/packagesamples/$PACKAGE_NAME/$PACKAGE_VERSION/$PACKAGE_NAME-$PACKAGE_VERSION.kxi
# We need to retrieve a INSIGHTS_BEARER token (this will live for 5 mins
# before having to be renewed)

# INSIGHTS_CLIENT_ID, INSIGHTS_CLIENT_SECRET provided from .env
INSIGHTS_BEARER=$(curl -X POST \
-s \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "client_id=$INSIGHTS_CLIENT_ID" \
-d "client_secret=$INSIGHTS_CLIENT_SECRET" \
-d "grant_type=client_credentials" \
"$INSIGHTS_URL/auth/realms/insights/protocol/openid-connect/token" | jq -
rc ".access_token")

# Push the package to the kdb Insights deploy
curl -X POST -H "Authorization: Bearer $INSIGHTS_BEARER"
$INSIGHTS_URL/packagemanager/v2/artifacts\?force=1 -F
"data=@${PACKAGE_NAME}-${PACKAGE_VERSION}.kxi"

# Deploy the package
curl -X POST -H "Authorization: Bearer $INSIGHTS_BEARER"
$INSIGHTS_URL/packagemanager/v2/deployments -H 'Content-Type:
application/json' -d '{
"data": {
"type":"deployment",
"attributes": {
"components": [
{
"type": "package",
"name": "'$PACKAGE_NAME'",
"version": "'$PACKAGE_VERSION'"
                }
            ]
        }
    }
}'

# Retrieve status
curl -X GET -H "Authorization: Bearer $INSIGHTS_BEARER"
$INSIGHTS_URL/packagemanager/v2/packages?
version=\&relationships=package,deployment,persistence

Next steps