Skip to content

Package Overview

This page provides an overview of kdb Insights Enterprise packages.

kdb Insights Enterprise packages provide a comprehensive solution by bundling tools and resources into a single, easy-to-use format. Packages streamline the management of various elements within the kdb Insights Enterprise platform. They simplify the complexities of data management by providing instructions on how to ingest, store, and query datasets, making the process accessible and actionable for those who require access.

Use the web interface or the CLI

kdb Insights Enterprise has both a web interface and command line interface (CLI). The web interface is a visual, browser-based tool for creating, querying, and visualizing data, while the (CLI) is a text-based tool for managing installations, configurations, and deployments programmatically. You can use either one to manage your packages.

Package structure

The package structure describes the set of files and sub-folders which represent components that control different aspects of the overall package.

The files and sub-folders for a newly created package look like the listing below:

$ kxi package init test-package
$ tree test-package
test-package
├── init.q
└── manifest.yaml

You can extend this to contain additional components. The following more complex package contains a database, a pipeline, and arbitrary code:

└──pkg-a
    ├── databases
       └── mydb
           ├── shards
              └── mydb-shard.yaml
           └── tables
    ├── init.q
    ├── manifest.yaml
    ├── pipelines
       └── pipe1.yaml
    └── src
        ├── init.q
        └── pipe1.q

Package manifest file

The manifest.yaml file serves as the packages index, describing configurable sections and dependencies. The file is crucial for the package's use by kdb Insights Enterprise.

When a package is created, the manifest.yaml file has the following structure.

kxi package -q init test-package --force && cat test-package/manifest.yaml
# yaml-language-server: $schema=https://code.kx.com/insights/enterprise/packaging/schemas/package.json#/$defs/Manifest
uuid: b4b5c232-084c-48b5-bd13-b8e4afa9c087
name: test-package
version: 0.0.1
metadata:
  description: ''
  authors:
  - {}
entrypoints:
  default: init.q

Editing the manifest file directly is not recommended, unless you are directed to do so.

The manifest is designed to reflect what is in the package. You should manipulate the manifest file through the kxi package tool, where possible. For example, if you change a pipeline name in the manifest and then run the kxi package refresh command, the manifest file changes the name back to the original name again.

Package manifest file sections

Do not delete any fields defined within the manifest.yaml file as part of the creation of a package!

The following table provides a brief description of each configurable section within the manifest.yaml and whether the definition of its content is required.

section description required
uuid The unique identifier for the package. yes
name The default name assigned to the package when building it. yes
version The default version number associated with the package when building it. yes
dependencies Any explicit dependencies on additional packages. For more information, refer to Package dependencies. no
metadata Information about the package contents and the users who have contributed to it. no
entrypoints The set of possible q/Python files used as the component for a package. For more information, refer to Package entrypoints. no
build Build, clean, and extra commands for a package.
system Information about the system conditions under which the package was generated, including the version of the CLI which was used to create the package. no
license The relative path to the license file under which the defined package is intended to be released. no
tables Tables linked to databases to deploy on kdb Insights Enterprise. no
databases The databases to deploy on kdb Insights Enterprise. no
pipelines Any Stream Processor pipeline definitions to deploy on kdb Insights Enterprise. no
routers Any Routers to deploy on kdb Insights Enterprise. no
reports Any Views to deploy on kdb Insights Enterprise. no
deployment_config Path to deployment_config to deploy on kdb Insights Enterprise. no
udf_namespaces This denotes the tagged names which are searched when parsing the package text for user-defined functions. no
patches Patches to apply to this package. no

Next steps