How to Add a Custom Client Package to Refinery

What is a client package used for?

A client package lets you upload your own custom code and run it in Refinery. This code works with both the API plugins and the internal process events. Use these features to tailor your Refinery installation to your own use cases without changing the Refinery core code.

Installing a client package

Installing a client package is similar to installing a Delta/Platform package. The simplest way to set one up is as follows:

Step 1 - connect to Control

Start the Refinery installation and connect to Control.

Step 2 - create new package

To create a new package, follow the steps below:

1. Navigate to the File tab.
2. Create a New package.
3. Enter a name, for example ClientPackage.
4. Ensure the Base Directory points to the installation's package directory.
5. Click Add.

Step 3 - create new user

The package must have an associated entity to install correctly. Create either a real or a dummy user, as follows:

1. Right-click the new package in the left-hand tab.
2. Click New > User.
3. Assign a name and password.
4. Click Add, then Save on the right-hand side.

Step 4 - export package

To export your package, follow these steps:

1. Open the Package Exporter by clicking the Tools tab on the top left of the screen.
2. Check the box for Include Permissions.
3. Click Export All.

This exports your package to your package directory (DELTA_PACKAGE_HOME) within your Refinery install.

Step 5 - set up folder structure

Add the following to your package: - install_config/profiles/ClientPackage.install.config - version.txt - A src directory (this is where your custom code is stored)

Rename the package to include a version number. The tree should look like this:

tree [DELTA_PACKAGE_HOME]/ClientPackage_0_0_1

├── ClientPackage.cfg
├── ClientPackage_export.cfg
├── ClientPackage_permissions.xml
├── install_config
│   └── profiles
│       └── ClientPackage.install.config
├── src
│   └── customCode.q
├── user
│   └── Dummy.xml
└── version.txt

Step 6 - add info to files

Add the following to their respective files:

ClientPackage.install.config

ENV|REF_CLIENT_PACKAGE|string|ClientPackage|0
DEP|

version.txt

product.name=ClientPackage
major.version=0
minor.version=0
incremental.version=1

Step 7 - tar package

Tar this package and add it to the Refinery package directory (KxRefinery(Finance)-5_X_X/packages/).

tar -czvf ~/KxRefinery(Finance)-5.X.X/packages/ClientPackage_0_0_1.tgz ClientPackage_0_0_1

Step 8 - add installation info

Add custom-environment-package-list=ClientPackage within the install.config that you use for installing/upgrading the Refinery.

Add ClientPackage to the DELTA_DAAS_SOLUTION_PACKAGES parameter within the install script (installKxPlatform.sh or installDeltaXML.sh).

Step 9 - run install script

Shut down your Refinery installation. Then, run the install script using either the Refinery CLI or the install script directly.

Step 10 - check installation was successful

When the installation is complete, open delta.profile and confirm that the installer added export REF_CLIENT_PACKAGE=ClientPackage.

Next, confirm that the source packages are in use. Query the .boot.cfg.srcPaths table from any process. A process picks up modified client files in the src directory when you restart it.

Alternative method: manual setup without the install script

The steps above use Control and the Refinery install script. Use that method for any package you want to keep across upgrades, because the install script also registers the package in custom-environment-package-list and DELTA_DAAS_SOLUTION_PACKAGES, so the package survives a later reinstall or upgrade.

To add or test a client package on an install that is already running, for example during development, you can build and register the package by hand instead. This is quicker, but the package isn't registered. Before you run a full install or upgrade, complete Step 8 above, or the installer ignores the package.

Step 1 - create the package directory

On the machine running the install, create the folder directly under the package directory, DELTA_PACKAGE_HOME. Include a version suffix in the name, for example MyClientPackage_0_0_1, and jot down the name as you'll need it later on. You don't need Control for this method.

mkdir -p [DELTA_PACKAGE_HOME]/MyClientPackage_0_0_1/src
mkdir -p [DELTA_PACKAGE_HOME]/MyClientPackage_0_0_1/install_config/profiles

The finished tree looks like this:

[DELTA_PACKAGE_HOME]/MyClientPackage_0_0_1
├── install_config
│   └── profiles
│       └── MyClientPackage.install.config
├── src
│   └── (your custom code, e.g. process/epdb.custom.q)
└── version.txt

Step 2 - add version.txt

Create [DELTA_PACKAGE_HOME]/MyClientPackage_0_0_1/version.txt containing:

product.name=MyClientPackage
major.version=0
minor.version=0
incremental.version=1

Step 3 - add the install-config file

Create [DELTA_PACKAGE_HOME]/MyClientPackage_0_0_1/install_config/profiles/MyClientPackage.install.config containing:

ENV|REF_CLIENT_PACKAGE|string|MyClientPackage|0
DEP|

This method doesn't run the install script, so nothing reads this file automatically. Keep it anyway. It records the environment variable that the package expects, and it lets you install the package with the script later.

Step 4 - add your custom code

Add your .q files to the src directory, for example src/process/epdb.custom.q. Name and structure each file to match the process type it attaches to.

Step 5 - set the environment variable

Instead of running the install script, edit delta.profile in the running install, usually at $DELTABIN_HOME/delta.profile. Add or update the following line:

export REF_CLIENT_PACKAGE=MyClientPackage

Only one client package can be active at a time. If the file already contains a REF_CLIENT_PACKAGE line from a previous client package, replace its value instead of adding a second line. Refinery matches the value as a prefix against the directory names in DELTA_PACKAGE_HOME, so MyClientPackage matches the MyClientPackage_0_0_1 directory from Step 1, whatever the version suffix.

Optionally, add a matching _HOME variable to make the package path available to your own code through getenv:

export MYCLIENTPACKAGE_VERSION=0_0_1
export MYCLIENTPACKAGE_HOME=${DELTA_PACKAGE_HOME}/MyClientPackage_${MYCLIENTPACKAGE_VERSION}

Save the file.

Step 6 - restart Delta Control and Refinery

Delta Control and the processes it runs read their environment only at startup. They don't re-read delta.profile while running, so restarting a single pipeline or process isn't enough. The already-running Delta Control starts that process and passes it the environment Delta Control itself started with.

To apply the change, stop the whole install:

refinery application --stop-all

Then restart it in the order described in the Quick Start Guide: Delta Control, Delta Control Daemon, core workflows, Process Manager, then pipelines.

This restarts every pipeline and process under the install, so plan for a full interruption.

Step 7 - verify

When the install is back up, confirm from any process that the package is active:

q).boot.cfg.srcPaths
q)getenv`REF_CLIENT_PACKAGE

The client row of .boot.cfg.srcPaths resolves inside MyClientPackage_0_0_1/src, and getenv returns `MyClientPackage.

From now on, when you edit files in the package's src directory, you only need to restart the affected process. A full restart is required only when the environment variables change.