Skip to content

Deploying a package

Before continuing

Make sure you have read through:

The deploy command enables you to deploy the database and pipeline configuration defined in your package onto kdb Insights Enterprise.

Once they are defined, you can push your package to kdb Insights Enterprise and run the deploy command to give the instruction to bring your configurations online.

Deploying

kxi package deploy --help

Usage: kxi package deploy [OPTIONS] SOURCE

  Deploy a package to an insights instance

Options:
  --with-version / --without-version
                                  Include the package's version in the
                                  deployment name. If --with-version is set
                                  the deployment will have a name like
                                  pkgname-100. If --without-version is set the
                                  deployment will have a name like pkgname.
                                  Note this is enabled by default to avoid
                                  ambiguity  [default: without-version]
  --remote / --local              Deploy a package from a remote kdb Insights
                                  Enterprise package repo (--remote)[default]
                                  or using a local package (--local) [WARN:
                                  'local' may be deprecated in future
                                  releases]
  --via [operator|controller]     Specify the deployment method. Available
                                  options: operator, controller
  --rm-existing-data              Remove the data associated with the old
                                  deployment
  --force                         Teardown running deployments if they share
                                  any properties with the package
  --db TEXT                       Deploy an existing package's database (must
                                  be defined in the package)
  --pipeline TEXT                 Deploy an existing package's pipeline (must
                                  be defined in the package)
  --url TEXT                      Insights URL[env var: INSIGHTS_URL; default:
                                  https://replace.me]
  --realm TEXT                    Realm[env var: INSIGHTS_REALM; default:
                                  insights]
  --client-id TEXT                Client id[env var: INSIGHTS_CLIENT_ID;
                                  default: test-publisher]
  --client-secret TEXT            Client secret[env var:
                                  INSIGHTS_CLIENT_SECRET; default: special-
                                  secret]
  --auth-enabled / --auth-disabled
                                  Will attempt to retrieve bearer token for
                                  request  [env var: KXI_AUTH_ENABLED]
  --help                          Show this message and exit.

Most of the parameters here are used to initialize a connection with kdb Insights Enterprise.

The deploy command can be run as follows:

kxi package deploy mypackage/1.0.0

This command searches the server for mypackage, version=1.0.0 and deploys the contents of that.

Code vs config

You may think of packages as being broadly in two categories:

  • Code: Packages that contain source code for other packages to leverage
  • Configuration: Packages that contain configurations for deployment

You can have a package with both code AND configuration however, deploying a package with only code in it will have no effect since the code is static and available to load at runtime only.

This means that code needs to be installed but config needs to be installed and then deployed. The config can reference any installed code, but that will not be run until deploy time.

The below diagram encapsulates the process of what happens when you request a deployment via the deploy command.

graph TD
    A(CLI) -->|Request Deployment| X(Package Manager Server);
    X --> B[Create Deployment Metadata object]
    B -->|Search by name for running entities| C{Found Running Entities ?};
    C -->|yes| D[Teardown Components by Name];
    D --> E{Was Teardown Successful?};
    E -->|yes| F[Send Deployment Payload to Backend];
    E -->|no| G[Fail];
    C -->|no| F;
    F --> J{Was Deployment Successful?};
    J -->|yes| H[Success]
    J -->|No| G;

--local flag

The local flag tells the deploy command to deploy from your local system, i.e. take a package it can find on the local computer and deploy that directly on kdb Insights Enterprise. It is in essence the same as the above except that the local client does all the work instead of the server.

This is not expected to be used for deploying production packages and could instead be viewed as a method to shorten the feedback loop when developing.

Tearing down

kxi package teardown --help

Usage: kxi package teardown [OPTIONS] DEPLOYMENT_REF

  Teardown a deployed packaged running on an insights instance

Options:
  --remote / --local              Teardown a deployment running on a kdb
                                  Insights Enterprise instance. This can be
                                  done by referencing a deployment by name and
                                  using the '--remote' [default]. Or by
                                  referencing a package in the local
                                  filesystem (--local) by name/ver [WARN:
                                  'local' may be deprecated in future
                                  releases]
  --via [operator|controller]     Specify the deployment method. Available
                                  options: operator, controller
  --rm-data                       Remove the data associated with the
                                  deployment
  --force                         Teardown running deployments if they share
                                  any properties with the package
  --url TEXT                      Insights URL[env var: INSIGHTS_URL; default:
                                  https://replace.me]
  --realm TEXT                    Realm[env var: INSIGHTS_REALM; default:
                                  insights]
  --client-id TEXT                Client id[env var: INSIGHTS_CLIENT_ID;
                                  default: test-publisher]
  --client-secret TEXT            Client secret[env var:
                                  INSIGHTS_CLIENT_SECRET; default: special-
                                  secret]
  --auth-enabled / --auth-disabled
                                  Will attempt to retrieve bearer token for
                                  request  [env var: KXI_AUTH_ENABLED]
  --help                          Show this message and exit.

The teardown command takes the package name that has been deployed and searches kdb Insights Enterprise for an assembly with that name. It will also match all the pipeline, database, stream and schema names it finds in the package and attempt to remove those too.

The force flag will attempt to remove entities more aggressively and will bypass some internal checks to see if the package is running or not.

Teardown can remove multiple running packages

Teardown will remove persistent volumes

--local Flag

The local flag tells the teardown command to teardown a package that has been deployed from your local system, it does this by resolving the local package before sending a request directly to the controller or operator.

Package Manager Server vs Controller vs Operator

The flow around deploy/teardown is more complex than the rest of the packaging subsystem. The usage of local and via are somewhat confusing without the context of what is happening at a system level.

This diagram aims to explain this:

graph LR
    A[CLI] -->|--remote| B(Package Manager Server);
    B -->|--via operator| D[operator];
    B -->|--via controller| C[controller];
    A ---->|--local --via controller| C;
    A ---->|--local --via operator| D;

The --via flag really controls which backend to deploy your package through. The benefit of using:

  • controller is that you get a UI representation.
  • operator is that it offers finer grain control (assuming you understand the underlying assembly).

The operator request can only be made with --local if you have kubernetes access to the kdb Insights Enterprise cluster on your local machine. The controller request relies only on your_kdb Insights Enterprise_ credentials and uses the kdb Insights Enterprise REST interface.

Tracking deployments

The deployment metadata objects that the deploy command creates aren't cleaned up by default. They are stored at KX_DEPLOYMENT_PATH and can be removed (but not toredown!) with kxi package uninstall all --obj-type=Deployment --force.

They are limited in scope and intended really only to track the lifecycle of "coming online" rather than health or status of the overall deployment (in their current scope).

In the future monitoring deployment state and health will be more complete.