Configure service classes in manifest.yaml¶

This page explains how to configure service classes per node in manifest.yaml, including port offsets, instance counts and thread settings; deploying instances of a service class is also covered.

In manifest.yaml, you set up your service classes. For each service class, you can define a base port offset, number of service class instances, number of threads, prefix, and taskset. Manifest information is defined separately for each node in a cluster, providing the flexibility for different service classes and counts across nodes.

Any changes to manifest.yaml after installation require that you import the changed file into etcd.

The following example shows a manifest.yaml file with a service class setup.

.default_service: &default_service
    enabled: true
    portOffset: 0
    num: 1
    threads: 0
    prefix: ""
    taskset: "0-10"

services:
  - sc: RTO
    <<: *default_service

  - sc: DBW
    <<: *default_service
    threads: 20

  - sc: SM
    <<: *default_service

  - sc: DBM
    <<: *default_service
    threads: 2

  - sc: RDB
    <<: *default_service
    num: 2
    threads: 4

  - sc: IDB
    <<: *default_service
    num: 2
    threads: 2

  - sc: HDB
    <<: *default_service
    num: 2
    threads: 2

  - sc: MDL
    <<: *default_service
    threads: 2

  - sc: SDL
    <<: *default_service
    num: 1
    threads: 4

  - sc: SGW
    <<: *default_service

  - sc: MON
    <<: *default_service

  - sc: HWMON
    <<: *default_service

  - sc: MONRDB
    <<: *default_service

The following table describes each attribute you can configure for a service class in manifest.yaml.

Attribute	Description
`svcClass`	Service class being set up.
`enabled`	Whether or not the service class is enabled; that is, automatically launched when the system starts up.
`portOffset`	Your base port for each service class. You can leave this column blank or you can assign a specific value. If left blank, the port will be set by the `KXS_BASE_PORT` environment variable in `kxsenv`; it will be some multiple of ten at the end of an already assigned port range. If you enter a value, it will be treated as an offset to your base port value `KXS_BASE_PORT` defined in `kxsenv`.
`num`	Number of services to run on a single node.
`threads`	Number of secondary threads that are available for parallel processing within the service. The default value is zero. Running on multiple cores gives parallelism to operations in Sensors processes that support it. It requires setting a secondary thread count greater than 1 and having a suitable kdb+ license to support this.
`prefix`	If required, you can assign a prefix to your service classes and add node and sequence information to a service class instance. You define system-generated names for service class instances in the `svcInstName` parameter in `systemParams.yaml`. For example, if your `svcInstName` = `{NODE}{SEQ}` and RDB is the first service class instance on node A, your RDB service class instance would be named `kxsRDB_A1`. For example, service class `DBW` with prefix `kxs` gives the full name `kxsDBW`; service class `HDB` with prefix `kxs` gives `kxsHDB`; service class `DA` with prefix `nx` gives `nxDA`.
`taskset`	CPU core or process affinity. The list of CPUs that this node/service class combination will have access to (for example, `1,2,8-11`). If you leave this column blank (the recommended option), the node/service class will have access to all CPUs. By default, only a single core is used for any given service class, and this core is selected dynamically by the operating system and changes over time.

Note

Explicitly specifying port ranges is only required if you wish to define firewall rules for service classes like RP, RTO or GW that communicate with external clients. When defining port ranges for these service classes, it is important to ensure that your port range is large enough to include all instances of the service class.

Example 1: portOffset = 0

RTO = 5000
DBW = 5020
SM = 5030

Example 2: portOffset = 200 for DBW

RTO = 5000
DBW = 5020
SM = 5220

General notes on ports and offsets:

When allocating ports for processes, RTO is allocated 50 ports.
Ports are allocated for service classes in the order in which they appear in the manifest.
The base port, base port + 1, and base port + 2 are always reserved for etcd, qetcd and etcd-peer port respectively.

Note

The DBW, DBC, DBM, GW, HWMON, MDL, MON, MONRDB, RTO, SGW, SM and CDL service classes are single instance only and will not accept numbers greater than 1.

Note

Assigning specific cores to specific processes is only recommended when required by kdb+ licensing restrictions.

Deploy instances of a service class¶

You can configure instances of a service class to run on one or more specific nodes of your system. If your service class is intended to always run, configure it in manifest.yaml and use kxsctl manifest import to import it into etcd. Service classes configured in manifest.yaml and imported into etcd start automatically. Service classes not added to manifest.yaml or not imported into etcd must be started manually in the kxs start <sc> script.

Understand port ranges¶

Consider the following examples:

.default_service: &default_service
    enabled: true
    portOffset: 0
    num: 1
    threads: 0
    prefix: ""
    taskset: "0-10"

services:
  - sc: RTO
    <<: *default_service

  - sc: GW
    <<: *default_service
    portOffset: 10

  - sc: RP
    <<: *default_service
    portOffset: 20
    num: 2
...

Assuming a base port of 5000, one might expect the first GW to be allocated port 5010. However, RTO appears first and is allocated ports 5003 to 5052. GW would then be allocated port 5053.

If you want to reserve port 5010 for GW and ports 5020 to 5199 to RPs, you can do this instead:

.default_service: &default_service
    enabled: true
    portOffset: 100
    num: 1
    threads: 0
    prefix: ""
    taskset: "0-10"

services:
  - sc: GW
    <<: *default_service
    portOffset: 10

  - sc: RP
    <<: *default_service
    portOffset: 20
    num: 2

  - sc: RTO
    <<: *default_service
...

Here we've set the default offset to be 100 and also ensured that the GW and RP entries precede any other.

Manage manifests with kxsctl¶

You manage the manifest held in etcd with the kxsctl manifest command.

Syntax	`kxsctl manifest show`, `kxsctl manifest export`, `kxsctl manifest import`, `kxsctl manifest edit`
Subcommands	`show` Prints the manifest as a table. `export <node> <file>` Writes the manifest to a YAML file. Use `-` for the file to write to stdout. `import <node> <file>` Replaces the manifest from a YAML file. Use `-` for the file to read from stdin. `edit <node>` Edits the manifest in place, either in your default editor or, with field flags, directly.

Changes in v3.3

kxsctl manifest replaces the kxsctl export and kxsctl import commands. Update any scripts that use the old commands.

Import your manifest file into etcd¶

When you import your manifest file using the kxsctl manifest import command, you assign your services, base port offset, number of service class instances, number of threads, prefix and taskset to a node. After performing an import, you must run kxsctl start for your changes to take effect.

If you wish to make any changes to manifest.yaml after your initial import, you must re-import your changes into etcd.

Examples

kxsctl manifest import A /root/kxs/src/manifest-n2.yaml
kxsctl manifest import B /root/kxs/src/manifest-n2.yaml

Edit a manifest in place¶

Instead of exporting, editing and re-importing, you can edit a manifest through kxsctl itself. With no field flags, kxsctl manifest edit <node> opens the manifest in your default editor. For scripting, kxsctl manifest edit <node> --sc <class> applies changes to one service class directly, using the following flags:

Flag	Description
`--enabled`	Whether the service class is enabled.
`--num`	Number of service class instances.
`--port-offset`	Base port offset.
`--prefix`	Instance name prefix.
`--threads`	Number of secondary threads.
`--taskset`	Cores to pin instances to.
`--tls-mode`	TLS mode for the service class.
`--auth`	Whether user authentication is required.

As with an import, you must run kxsctl start for your changes to take effect.

Export your manifest file from etcd¶

You can export a manifest file from etcd to a file location in your local drive for editing purposes. After making your changes locally, you must re-import it into etcd before your changes will take effect.

Syntax	`kxsctl manifest export <node> <file>`
Arguments	`<node>` The node whose manifest you want to export. `<file>` The file location on your local drive where you want to save the exported manifest. Use `-` to write to stdout.

Example 1 (printing to the Console)¶

[root@kxsa /]# kxsctl manifest export A - | head
# Exported 19 Services
services:
    - sc: RTO
      enabled: true
      portOffset: 0
      num: 1
      prefix: ""
      threads: 1
      taskset: 0-10

    - sc: DBW

Example 2 (printing to file)¶

[root@kxsa /]# kxsctl manifest export A myManifest.yaml
# Exported 19 Services
[root@kxsa /]# cat myManifest.yaml | head
services:
    - sc: RTO
      enabled: true
      portOffset: 0
      num: 1
      prefix: ""
      threads: 1
      taskset: 0-10

    - sc: DBW
      enabled: true

Next steps¶

Add a node
Set up a message queue
Set up your EMS streams in ems.yaml
Call APIs over REST — enable REST support on SGW