Deploying

Before Continuing

Make sure you've read through:

The deploy command enables us to deploy our database & pipeline configuration fined in our package onto kdb Insights Enterprise.

Once they are defined, we can push our package to kdb Insights Enterprise and run the deploy command to give the instruction to bring our configurations online.

Deploying

kxi package deploy --help

Options: --force --db TEXT --help

Usage: kxi package deploy [OPTIONS] SOURCE Deploy a package to an insights instance --with-version / --without-version Include the package's version in the deployment name. If --with-version is set the deployment will have a name like pkgname-100. If --without-version is set the deployment will have a name like pkgname. Note this is enabled by default to avoid ambiguity  [default: without-version] --remote / --local              Deploy a package from a remote kdb Insights Enterprise package repo (--remote)[default] or using a local package (--local) [WARN: 'local' may be deprecated in future releases] --via [operator|controller]     Specify the deployment method. Available options: operator, controller --rm-existing-data              Remove the data associated with the old deployment Teardown running deployments if they share any properties with the package Deploy an existing package's database (must be defined in the package) --pipeline TEXT                 Deploy an existing package's pipeline (must be defined in the package) --url TEXT                      Insights URL[env var: INSIGHTS_URL; default: https://replace.me] --realm TEXT                    Realm[env var: INSIGHTS_REALM; default: insights] --client-id TEXT                Client id[env var: INSIGHTS_CLIENT_ID; default: test-publisher] --client-secret TEXT            Client secret[env var: INSIGHTS_CLIENT_SECRET; default: special- secret] --auth-enabled / --auth-disabled Will attempt to retrieve bearer token for request  [env var: KXI_AUTH_ENABLED] Show this message and exit. class="markdown-exec">


Most of the parameters here are used to initialise a connection with kdb Insights Enterprise.


The deploy command can be run like:
kxi package deploy mypackage/1.0.0

Which will search the server for mypackage, version=1.0.0 and deploy the contents of that.

Code vs Config
You may think of packages as being broadly in two categories:

Code: Packages that contain source code for other packages to leverage
Config: Packages that contain configurations for deployment

Of course there is no limitation and we can have a package with both code AND config, however, deploying a package with only code in it will have no effect since the code is static and available to load at runtime only.
So code, needs installed but config needs installed then deployed. 
The config can reference any installed code, but that wont be run until deploy time.

The below diagram encapsulates the process of what happens when you request a deployment via the deploy command.
graph TD
    A(CLI) -->|Request Deployment| X(Package Manager Server);
    X --> B[Create Deployment Metadata object]
    B -->|Search by name for running entities| C{Found Running Entities ?};
    C -->|yes| D[Teardown Components by Name];
    D --> E{Was Teardown Successful?};
    E -->|yes| F[Send Deployment Payload to Backend];
    E -->|no| G[Fail];
    C -->|no| F;
    F --> J{Was Deployment Successful?};
    J -->|yes| H[Success]
    J -->|No| G;

--local Flag
The local flag tells the deploy command to deploy from your local system, i.e. take a package it can find on the local computer and deploy that directly on kdb Insights Enterprise.
It is in essence the exact same as the above except that the local client does all the work instead of the server.
This is not expected to be used for deploying production packages and could instead be viewed as a method to shorten the feedback loop when developing.

Tearing down

kxi package teardown --help
Usage: kxi package teardown [OPTIONS] DEPLOYMENT_REF

  Teardown a deployed packaged running on an insights instance

Options:
  --remote / --local              Teardown a deployment running on a kdb
                                  Insights Enterprise instance. This can be
                                  done by referencing a deployment by name and
                                  using the '--remote' [default]. Or by
                                  referencing a package in the local
                                  filesystem (--local) by name/ver [WARN:
                                  'local' may be deprecated in future
                                  releases]
  --via [operator|controller]     Specify the deployment method. Available
                                  options: operator, controller
  --rm-data                       Remove the data associated with the
                                  deployment
  --force                         Teardown running deployments if they share
                                  any properties with the package
  --url TEXT                      Insights URL[env var: INSIGHTS_URL; default:
                                  https://replace.me]
  --realm TEXT                    Realm[env var: INSIGHTS_REALM; default:
                                  insights]
  --client-id TEXT                Client id[env var: INSIGHTS_CLIENT_ID;
                                  default: test-publisher]
  --client-secret TEXT            Client secret[env var:
                                  INSIGHTS_CLIENT_SECRET; default: special-
                                  secret]
  --auth-enabled / --auth-disabled
                                  Will attempt to retrieve bearer token for
                                  request  [env var: KXI_AUTH_ENABLED]
  --help                          Show this message and exit.



The teardown command takes the package name that has been deployed and searches Insights for an assembly with that name. 
It will also match all the pipeline, database, stream and schema names it finds in the package and attempt to remove those too.
The force flag will attempt to remove entities more aggressively and will bypass some internal checks to see if the package is running or not.

Teardown can remove multiple running packages


Teardown will remove persistent volumes


--local Flag
The local flag tells the teardon command to teardown a package that has been deployed from your local system, it does this by resolving the local package before sending a request directly to the controller or operator.

Package Manager Server vs Controller vs Operator
The flow around deploy/teardown is more complex than the rest of the packaging subsystem. 
The usage of local and via are somewhat confusing without the context of what is happening at a system level.
This diagram aims to explain this:
graph LR
    A[CLI] -->|--remote| B(Package Manager Server);
    B -->|--via operator| D[operator];
    B -->|--via controller| C[controller];
    A ---->|--local --via controller| C;
    A ---->|--local --via operator| D;
The --via flag really controls which backend to deploy our package through.
The benefit of using:

controller is that we get a UI representation.
operator is that it offers finer grain control (assuming you understand the underlying assembly).

The operator request can only be made with --local if you have kubernetes access to the Insights cluster on your local machine.
The controller request relies only on your kdb Insights Enterprise credentials and uses the kdb Insights Enterprise REST interface.
Tracking deployments
The deployment metadata objects that the deploy command creates aren't cleaned up by default.
They are stored at KX_DEPLOYMENT_PATH and can be removed (but not toredown!) with kxi package uninstall all --obj-type=Deployment --force.
They are limited in scope and intended really only to track the lifecycle of "coming online" rather than health or status of the overall deployment (in their current scope).
In the future we will make monitoring deployment state and health more complete.