Skip to content

Package Overview

This page provides an overview of kdb Insights Enterprise packages.

kdb Insights Enterprise packages provide a comprehensive solution by bundling tools and resources into a single, easy-to-use format. Packages streamline the management of various elements within the kdb Insights Enterprise platform. They simplify the complexities of data management by providing instructions on how to ingest, store, and query datasets, making the process accessible and actionable for those who require access.

Use the web interface or the CLI

kdb Insights Enterprise has both a web interface and command line interface (CLI). The web interface is a visual, browser-based tool for creating, querying, and visualizing data, while the (CLI) is a text-based tool for managing installations, configurations, and deployments programmatically. You can use either one to manage your packages.

Package structure

The package structure describes the set of files and sub-folders which represent components that control different aspects of the overall package.

The files and sub-folders for a newly created package look like the listing below:

$ kxi package init test-package
$ tree -a test-package
test-package
├── .kxignore
└── manifest.yaml

Initialization creates only these two files. It does not create init.q or any other source file; you add source files and entrypoints yourself after initialization.

You can extend this to contain additional components. The following more complex package contains a database, a pipeline, and arbitrary code, where init.q and the files under src have been added by the package author:

└──pkg-a
    ├── databases
    │   └── mydb
    │       ├── shards
    │       │   └── mydb-shard.yaml
    │       └── tables
    ├── init.q
    ├── manifest.yaml
    ├── pipelines
    │   └── pipe1.yaml
    └── src
        ├── init.q
        └── pipe1.q

Package manifest file

The manifest.yaml file serves as the packages index, describing configurable sections and dependencies. The file is crucial for the package's use by kdb Insights Enterprise.

When a package is created, the manifest.yaml file has the following structure.

kxi package -q init test-package --force && cat test-package/manifest.yaml

# yaml-language-server: $schema=https://code.kx.com/insights/enterprise/packaging/schemas/package.json#/$defs/Manifest
metadata: {}
name: test-package
version: 0.0.1
dependencies: []
tables: {}
databases: {}
pipelines: {}
views: {}
patches: []

Editing the manifest file directly is not recommended, unless you are directed to do so.

The manifest is designed to reflect what is in the package. You should manipulate the manifest file through the kxi package tool, where possible. For example, if you change a pipeline name in the manifest and then run the kxi package refresh command, the manifest file changes the name back to the original name again.

Package manifest file sections

Do not delete any fields defined within the manifest.yaml file as part of the creation of a package!

The following table provides a brief description of each configurable section within the manifest.yaml and whether the definition of its content is required.

section description required
name The default name assigned to the package when building it. yes
version The default version number associated with the package when building it. yes
version_info A description of the changes made in this version of the package. no
dependencies Any explicit dependencies on additional packages. For more information, refer to Package Dependency. no
metadata Information about the package contents and the users who have contributed to it. no
entrypoints The set of possible q/Python files used as the component for a package. For more information, refer to Package entrypoints. no
build Build, clean, and extra commands for a package. no
license The relative path to the license file under which the defined package is intended to be released. no
tables Tables linked to databases to deploy on kdb Insights Enterprise. no
databases The databases to deploy on kdb Insights Enterprise. no
pipelines Any Stream Processor pipeline definitions to deploy on kdb Insights Enterprise. no
router The path to the package Router file to deploy on kdb Insights Enterprise; for example router/router.yaml. no
views Any Views to deploy on kdb Insights Enterprise. The name reports is still accepted when reading an existing package. no
deployment_config Path to deployment_config to deploy on kdb Insights Enterprise. no
udf_namespaces This denotes the tagged names which are searched when parsing the package text for user-defined functions. no
patches Patches to apply to this package. no

Next steps

Back to top