Automated Package Deployment
This page outlines how packages can be deployed automatically with CI/CD processing.
The recommended approach is to use the CLI for automated deployments.
However, as kdb Insights Enterprise is OpenAPI-compliant with support API endpoints for all package operations, the same operations can be performed through API calls by any REST-compliant client.
KXI CLI
The KXI CLI is the primary means to deploy packages.
This is a command line tool built on Python which offers zero prompt interactions with packages in all APIs.
Read Install KXI CLI for information on installing the CLI.
CLI prerequisites
Object | Format |
---|---|
client-id | $INSIGHTS_CLIENT_ID |
client-secret | $INSIGHTS_CLIENT_SECRET |
Package file | $PACKAGE_NAME-$PACKAGE_VERSION.kxi |
Package location | for example KX portal |
Ensure the following:
-
You are running kdb Insights Enterprise 1.12+
-
You have created a client capable of authenticating to Insights
-
You have created a client that has the necessary roles and entitlements to deploy packages
-
You have published the package and made it available in a repository accessible from a REST client
-
You have curl available on the machine performing the deployment
-
KXI CLI v1.12 or later installed and available on the machine performing the deployment
-
KXI CLI configured to the Insights deploy
Steps using CLI
This example uses a package called kxi-db
on package version 1.12.0.
# Pull in secrets. This could be from a secure secrets store or an .env
# only accessible to the automated system BEARER (package repository),
# INSIGHTS_CLIENT_ID (insights client name), INSIGHTS_CLIENT_SECRET source .env
# Package Details
PACKAGE_NAME="kxi-db"
PACKAGE_VERSION="1.0.0"
# Pull the package from the package repository
# BEARER provided from .env
curl -s -L -OJ --fail-with-body -D /dev/stderr --oauth2-bearer $BEARER
https://portal.dl.kx.com/assets/raw/packagesamples/$PACKAGE_NAME/$PACKAGE_VERSION/$PACKAGE_NAME-$PACKAGE_VERSION.kxi
# Push the package to the Insights deploy
# INSIGHTS_CLIENT_ID, INSIGHTS_CLIENT_SECRET provided from .env
kxi pm push --client-id=$INSIGHTS_CLIENT_ID --client-secret=$INSIGHTS_CLIENT_SECRET --force
${PACKAGE_NAME}-${PACKAGE_VERSION}.kxi
# Deploy the package
kxi pm deploy --client-id=$INSIGHTS_CLIENT_ID --client-secret=$INSIGHTS_CLIENT_SECRET ${PACKAGE_NAME}
# Retrieve status
kxi pm list --client-id=$INSIGHTS_CLIENT_ID --client-secret=$INSIGHTS_CLIENT_SECRET ${PACKAGE_NAME} -o json
You can use the help command to provide information regarding the additional options
Usage: kxi pm deploy [OPTIONS] PACKAGE [VERSION]
Deploy a package to an insights instance.
PACKAGE: package-name VERSION: package-version
╭─ Authentication option overrides ────────────────────────────────────────────╮
│ --hostname,--url TEXT Insights URL │
│ --realm TEXT Realm │
│ --client-id TEXT Client id │
│ --client-secret TEXT Client secret │
│ --auth-enabled/--auth-disabled Retrieve Bearer Token │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --env TEXT Inject environment variables to the │
│ deployed package. │
│ `[component_name:]VAR=value` │
│ --output-format -o [json|table] Output format for the list command │
│ --server-timeout TEXT Timeout for Insights server calls │
│ --help Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
curl
The OpenAPI endpoints can be used for all operations, allowing any REST-compliant client to perform the same operations as KXI CLI.
curl prerequisites
Object | Format |
---|---|
client-id | $INSIGHTS_CLIENT_ID |
client-secret | $INSIGHTS_CLIENT_SECRET |
Package file | $PACKAGE_NAME-$PACKAGE_VERSION.kxi |
Package location | For example KX portal |
Ensure the following:
- You are running kdb Insights Enterprise 1.12+
- You have created a client capable of authenticating to Insights
- You have created a client that has the necessary roles and entitlements to deploy packages
- You have published the package and made it available in a repository accessible from a REST client
- You have curl available on the machine performing the deployment
Steps using curl
-
Create a service account to permit authentication.
-
Specify and save your client-id and client-secret.
-
Add authorizations to permit pushing or deploying packages: assign "insights.role.maintainer" role.
-
Locate the required package and make it available to a pipeline in a repository accessible from a REST client.
-
Create curl script to:
- Pull package from repository; for example Git, Nexus, portal.dl.kx.com etc
- Push and deploy package to kdb Insights Enterprise
- Retrieve status of package
# Pull in secrets. This could be from a secure secrets store or an .env
# only accessible to the automated system BEARER (package repository),
INSIGHTS_CLIENT_ID (insights client name), INSIGHTS_CLIENT_SECRET
source .env
# Package Details
PACKAGE_NAME="kxi-db"
PACKAGE_VERSION="1.0.0"
# kdb Insights URL - please update to accessible endpoint.
#Example:
INSIGHTS_URL="https://insights.domain.com"
# Pull the package from the package repository
# BEARER provided from .env
curl -s -L -OJ --fail-with-body \-D /dev/stderr --oauth2-bearer $BEARER
https://portal.dl.kx.com/assets/raw/packagesamples/$PACKAGE_NAME/$PACKAGE_VERSION/$PACKAGE_NAME-$PACKAGE_VERSION.kxi
# We need to retrieve a INSIGHTS_BEARER token (this will live for 5 mins
# before having to be renewed)
# INSIGHTS_CLIENT_ID, INSIGHTS_CLIENT_SECRET provided from .env
INSIGHTS_BEARER=$(curl -X POST \
-s \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "client_id=$INSIGHTS_CLIENT_ID" \
-d "client_secret=$INSIGHTS_CLIENT_SECRET" \
-d "grant_type=client_credentials" \
"$INSIGHTS_URL/auth/realms/insights/protocol/openid-connect/token" | jq -
rc ".access_token")
# Push the package to the kdb Insights deploy
curl -X POST -H "Authorization: Bearer $INSIGHTS_BEARER"
$INSIGHTS_URL/packagemanager/v2/artifacts\?force=1 -F
"data=@${PACKAGE_NAME}-${PACKAGE_VERSION}.kxi"
# Deploy the package
curl -X POST -H "Authorization: Bearer $INSIGHTS_BEARER"
$INSIGHTS_URL/packagemanager/v2/deployments -H 'Content-Type:
application/json' -d '{
"data": {
"type":"deployment",
"attributes": {
"components": [
{
"type": "package",
"name": "'$PACKAGE_NAME'",
"version": "'$PACKAGE_VERSION'"
}
]
}
}
}'
# Retrieve status
curl -X GET -H "Authorization: Bearer $INSIGHTS_BEARER"
$INSIGHTS_URL/packagemanager/v2/packages?
version=\&relationships=package,deployment,persistence