Skip to content

How to Manage Workers in KDB.AI

This page provides details on how to add, verify, and remove workers in KDB.AI.

If you're new to this topic, start with Learn: Parallel Processing.

Workers provide additional processing capacity for concurrent workloads. You can add workers to increase processing capacity or remove them when additional capacity is no longer required.

Before you start

Ensure you have:

Add a worker

Add a worker to increase processing capacity. Specify one of the following worker types:

Worker type When to use it
general Use for user workloads such as ingestion, search, and queries.
internal Use for internal system and administrative operations.

Note

Currently, only Info API operations run on internal workers. Ingestion, search, queries, and all other user workloads run on general workers.

License requirement

Adding a new worker through addWorker requires a k4 license. If the runtime is in vdbRestricted mode, the call is rejected with:

KDBAI-SRT-SW-001 | adding new worker is not allowed in this license
Upgrade to a k4 license to enable multi-worker support.

When the operation completes successfully, KDB.AI returns a Worker ID. Save this ID if you plan to remove the worker later.

session = kdbai.Session(oauth=<oauth_config_details>)
session.admin.add_worker(type="general")

// Open connection
`gw set hopen `::8082:<username>:<oauth_token>;

// Add worker
gw(`addWorker;enlist[`type]!enlist `general)

curl -H 'Content-Type: application/json' -H 'Authorization: Bearer <token>' localhost:8081/api/v2/admin/workers -d '{"type" : "general"}'

Verify worker status

After adding a worker, use getProcessInfo (or session.process_info() in Python) to verify that the new worker is running. getProcessInfo returns information about all running workers, including their worker IDs and current status.

session.process_info()

gw(`getProcessInfo;`)

curl -H 'Content-Type: application/json' -H 'Authorization: Bearer <token>' localhost:8081/api/v2/info/process

For more details about the returned process information, refer to How to Get System Information.

Remove a worker

Remove a worker when additional processing capacity is no longer required. Specify the Worker ID returned when the worker was created.

The remove worker operation applies only to general workers. Internal workers cannot be removed.

session = kdbai.Session(oauth=<oauth_config_details>)
session.admin.remove_worker(id="worker-general-2-8c6b8b64-6815-6084-0a3f-11080180fce9")

// Open connection
`gw set hopen `::8082:<username>:<oauth_token>;

// Remove worker
gw(`removeWorker;enlist[`id]!enlist `worker-general-2-8c6b8b64-6815-6084-0a3f-11080180fce9)

curl -X DELETE -H 'Content-Type: application/json' -H 'Authorization: Bearer <token>' localhost:8081/api/v2/admin/workers/worker-general-2-8c6b8b64-6815-6084-0a3f-11080180fce9

Troubleshooting

Issue Resolution
Invalid worker type Specify either general or internal.
Worker not found Verify that the Worker ID is correct.
Unable to connect to KDB.AI Server Verify that KDB.AI Server is running and that the host, port, and authentication details are correct.

Next steps

Now that you've learned how to manage workers, you can: