Skip to content

OAuth 2.0 Authentication and Authorization in KDB.AI

This page explains how to integrate KDB.AI with an external identity provider (IdP) such as Keycloak, Microsoft Entra ID, or Okta to enable OAuth 2.0-based access control. It covers JWT token validation, required environment configuration, tenant and group-based authorization using ACL grants, and Python client integration.

If you're new to this topic, start with Learn: Authentication.

Prerequisites

Before configuring OAuth 2.0, you need:

  • An IdP that supports OAuth 2.0 and JWTs
  • At least one tenant defined in your IdP
  • An OAuth 2.0 client for each tenant
  • Group membership defined in your IdP
  • Ability to modify environment variables for KDB.AI

How it works

KDB.AI uses the OAuth 2.0 framework to authorize requests based on JWT access tokens issued by your identity provider (IdP). It supports JWT access tokens only and validates their signatures using the RS256 (RSA with SHA-256) algorithm. Public signing keys are retrieved from the issuer’s JWKS endpoint. After signature validation, KDB.AI evaluates the tenant, groups (with claim field names defined by the user), and aud claims and matches them against stored ACL grants to determine whether the request is permitted.

Identity provider responsibility

All user management, authentication, and token issuance remain with your IdP. KDB.AI does not manage users, passwords, or sessions, and does not communicate with the IdP beyond fetching JWKS keys for signature verification.

OAuth 2.0 Authorization Flow (REST API)

Identity Provider → issues JWT → Client → sends Bearer token → KDB.AI
Step What happens
1. Token request Client obtains a JWT token from the IdP.
2. API request Client sends the JWT as a Bearer token in the Authorization header of the REST call.
3. Signature check KDB.AI fetches JWKS keys from the issuer URL and validates the token signature using RS256.
4. Claim validation KDB.AI checks: iss is in the allowed issuers list, aud matches the expected app name.
5. Claim extraction KDB.AI reads the tenant and groups claims from the token.
6. ACL evaluation KDB.AI matches (tenant, groups) against stored grants to authorize the request.

sequenceDiagram
    participant C as Client
    participant IdP as Identity Provider
    participant K as KDB.AI

    C->>IdP: 1. Token request
    IdP-->>C: JWT (RS256)

    C->>K: 2. API request with Bearer token

    Note right of K: 3. Validate JWT signature (RS256)

    Note right of K: 4. Validate claims (iss, aud, exp)

    Note right of K: 5. Extract tenant and groups

    Note right of K: 6. Evaluate ACL grants

    K-->>C: Allow (200) or Deny (401/403)

For Python client integration, refer to the KDB.AI Python Client section. You can configure the client to automatically handle OAuth 2.0 token refresh and lifecycle management.

Example IdP structure

The tenant and group names below are for demo purposes only. Replace them with your own organizational structure. Users are shown for IdP context only.

KDB.AI never sees or stores user identities. It only sees the tenant and groups claims in the token.

Tenant Groups Users (IdP only) Group membership
quants admin, trader, viewer alice trader, viewer
bob viewer
risk admin, viewer charlie viewer
manager admin root admin

Each tenant is an isolated boundary. Groups within a tenant have no inherent meaning on their own – they only gain significance when a KDB.AI ACL grant references them (refer to Group-to-Permission Mapping).

IdP terminology

How a "tenant" is represented depends on your IdP:

  • In Keycloak, each tenant is a realm.
  • In Entra ID, it maps to an Entra tenant (identified by tid).
  • In Okta, it could be an org or authorization server.

In KDB.AI, the user can configure the tenant field name.

Required JWT claims

The IdP must include the following claims in the access token through claim mappings (or equivalent configuration in your IdP). KDB.AI uses these to validate the token and authorize each request.

Claim Example value IdP configuration Purpose
iss https://idp.example.com/tenants/quants Standard claim (set automatically by the IdP) Must match one of the OAUTH_ISSUERS entries. Also used to fetch JWKS keys for signature verification.
aud kdbai-service Audience mapper Must match OAUTH_CLIENT_ID. Prevents tokens intended for other services from being accepted.
tenant quants Hardcoded claim mapper Identifies which tenant the token belongs to. Claim name is configurable using OAUTH_TENANT_CLAIM.
groups ["trader", "viewer"] Group membership mapper List of group memberships. Matched against ACL grants to determine what operations the token authorizes. Claim name is configurable using OAUTH_GROUPS_CLAIM.

Identity provider setup

Regardless of which IdP you use, you need at least one OAuth 2.0 client per tenant. Each client gets its own claim mappings with a hardcoded tenant value identifying that tenant. A tenant can have multiple clients – for example, a public client for interactive users (authorization code, device authorization) and a confidential client for machine-to-machine access (client credentials). All clients for the same tenant share the same tenant claim value.

Microsoft Entra ID

For end-to-end setup instructions with Microsoft Entra ID, see the KDB.AI OAuth 2.0 with Microsoft Entra ID guide.

For each tenant:

  1. Create the OAuth 2.0 client in your IdP.

  2. Add three claim mappings to the client:

    • Audience → sets aud to your client ID (for example, kdbai-service).
    • Groups → emits the user's group memberships as a groups claim.
    • Tenant → emits a hardcoded tenant claim identifying this specific tenant.
  3. Create groups that correspond to the group names you will use in KDB.AI ACL grants.

  4. Assign users to groups.

Groups and users

Groups in your IdP are the bridge between users and KDB.AI permissions. A group name on its own does nothing – it only gains meaning when a KDB.AI ACL grant references that (tenant, group) pair.

For example:

  • Creating a group called trader in the quants tenant does not automatically grant any access.
  • Only when a KDB.AI system_admin creates a grant like "give trader in quants read access to database analytics" does the group have an effect.
  • Any user in the trader group will then inherit that access.

Create groups in each tenant that reflect the access levels you want to define, then assign users to those groups. A user can belong to multiple groups and will receive the union of all matching grants.

KDB.AI configuration

The kdbai‑db container requires the following environment variables to enable OAuth 2.0 token validation and authorization. Without them, the system disables access control and treats all requests as anonymous.

Variable Required Example Description
AUTH_TYPE Yes oauth Enables JWT-based authorization
OAUTH_ISSUERS Yes View format Comma-separated string of trusted token issuer URLs – one per tenant
OAUTH_CLIENT_ID Yes kdbai-service The client ID of the OAuth 2.0 application registered in your IdP. Must match the aud claim in tokens issued for KDB.AI
OAUTH_TENANT_CLAIM Yes tenant The name of the JWT claim KDB.AI reads to identify the token's tenant. For example, if the token contains "tenant": "quants", set this to tenant
OAUTH_GROUPS_CLAIM Yes groups The name of the JWT claim KDB.AI reads to identify the token's group memberships. For example, if the token contains "groups": ["trader", "viewer"], set this to groups
ACL_SYSTEM_ADMIN_TENANT Yes manager The identifier of the tenant that the system admin group belongs to. Can be a name or ID depending on your IdP. Refer to System admin bootstrap
ACL_SYSTEM_ADMIN_GROUP Yes admin The identifier of the group whose members will have system admin privileges in KDB.AI. Can be a name or ID depending on your IdP. Refer to System admin bootstrap

ACL data persistence

ACL grants are stored separately from VDB data at a different path inside the container. Without a dedicated volume mount, all grants are lost when the container is removed or recreated.

Path Volume mount
/tmp/acl ./acl-data:/tmp/acl

Important

This volume is separate from the VDB data volume (/tmp/kx/data). If you mount only the VDB volume and not the ACL volume, your data persists, but the system must recreate all ACL grants after every container restart. In production, you should mount both volumes.

System admin bootstrap

ACL_SYSTEM_ADMIN_TENANT and ACL_SYSTEM_ADMIN_GROUP define which (tenant, group) pair gets system_admin privileges at startup. These values can be anything – manager and admin are just example names. They must match a real tenant and group in your IdP.

flowchart TD
    ENV[ACL_SYSTEM_ADMIN_TENANT + ACL_SYSTEM_ADMIN_GROUP] --> MATCH{Token tenant & group match?}

    MATCH -- Yes --> ADMIN[system_admin access<br/>Full privileges]
    MATCH -- No --> NORMAL[Standard ACL evaluation]

Scenario What happens
Both set and match a token's claims Any token with that (tenant, group) pair gets full admin access: manage grants, bypass all ACL checks, access all resources
Not set (empty or missing) Container fails to start. The gateway throws: "need ACL_SYSTEM_TENANT_GROUP env variable value"
Set to values that don't match any token Container starts, but no token has system_admin access. No one can call addGrants or deleteGrant, so you are locked out of ACL management.

Tip

Always verify that your IdP can issue tokens with the configured tenant and group claims before starting KDB.AI with OAuth enabled. If you get locked out, fix the environment variables and restart – no data is lost.

Issuer URL format

OAUTH_ISSUERS accepts a comma-separated list of issuer URLs. Each entry must exactly match the iss claim in the tokens issued by that IdP. Add one entry per tenant that should be allowed to authenticate.

Example

OAUTH_ISSUERS=https://idp.example.com/tenants/quants,https://idp.example.com/tenants/risk,https://idp.example.com/tenants/manager

Important

These URLs must be reachable from the KDB.AI container at runtime. KDB.AI uses each issuer URL to fetch the OpenID Connect discovery document, which in turn provides the JWKS endpoint used to retrieve signing keys. If a URL is unreachable, tokens from that issuer will fail validation.

Example configuration

The following Docker Compose example shows all required environment variables and volume mounts. The same configuration applies to any deployment method (for example, Kubernetes, standalone Docker). Adapt the environment variables and volume mounts to your platform.

services:
  kdbai-db:
    image: portal.dl.kx.com/kdbai-db:latest
    environment:
      - KDB_LICENSE_B64=${KDB_LICENSE_B64}               # base64 encoded license string
      # --- OAuth (required) ---
      - AUTH_TYPE=oauth
      - OAUTH_CLIENT_ID=<oauth-app-name>                  # must match the "aud" claim in your tokens
      - OAUTH_TENANT_CLAIM=<tenant-claim-name>              # name of the JWT claim that holds the tenant ID
      - OAUTH_GROUPS_CLAIM=<groups-claim-name>              # name of the JWT claim that holds the groups ID
      - OAUTH_ISSUERS=<token-issuer-url>                 # token endpoint - supports comma separate string
      # --- System admin (required, see "System admin bootstrap" above) ---
      - ACL_SYSTEM_ADMIN_TENANT=<system-admin-tenant-id> # The identifier of the tenant that the system admin group belongs to
      - ACL_SYSTEM_ADMIN_GROUP=<system-admin-group-id>   # The identifier of the group whose members will have system admin privileges
    volumes:
      - ./kdbai-data:/tmp/kx/data                        # kdbai vdb data
      - ./acl-data:/tmp/acl                              # persists ACL grants across restarts
    ports:
      - 8082:8082                                        # QIPC port
      - 8081:8081                                        # REST port

Update the OAUTH_ISSUERS to match your IdP configuration. The issuer URL must exactly match the iss claim in your tokens. For example:

  • Keycloak: https://<keycloak-host>/realms/<realm> – one entry per tenant (realm)
  • Entra ID: https://login.microsoftonline.com/<tenant-id>/v2.0 (v2, recommended) or https://sts.windows.net/<tenant-id>/ (v1) — see Token API version for details
  • Okta: https://<your-domain>.okta.com/oauth2/<authorization-server-id>

Group-to-permission mapping

Permissions in KDB.AI are not assigned to individual users. They are assigned to (tenant, group) pairs through grants. When a request is received, KDB.AI extracts tenant and groups from the JWT and checks whether any grant matches.

How groups connect to permissions

You define groups in your IdP and create grants in KDB.AI that reference those same group names. Apart from fetching JWKS keys to verify token signatures, KDB.AI never talks to your IdP - it only reads the group names from the JWT. If a grant exists for that (tenant, group) pair, the request is authorized. If not, it is denied.

IdP                                              KDB.AI
========================                         ======

Tenant: "quants"                                 ACL Grant:
  Group: "trader"            --- maps to --->      tenant=quants, groups=[trader]
    User: alice                                    database=analytics, actions=[read, write]
    User: dave

Tenant: "quants"                                 ACL Grant:
  Group: "viewer"            --- maps to --->      tenant=quants, groups=[viewer]
    User: alice                                    database=analytics, actions=[read]
    User: bob

The group name in the IdP must exactly match the group name in the KDB.AI grant. The grant is what gives the group its meaning.

Access levels

Level Includes Allowed operations
system_admin everything Full access: manage grants, bypass all ACL checks, access all resources across all tenants. This level is not assigned through grants – it is determined by the ACL_SYSTEM_ADMIN_TENANT and ACL_SYSTEM_ADMIN_GROUP environment variables (refer to System admin bootstrap). Any token whose tenant and groups claims match these values automatically has system_admin privileges.
delete read Drop table, drop database, delete data
write read Insert data, create table, update data, update indexes
read Query, search, get table, list tables, get index, list indexes

Note

write includes read. delete includes read. But delete does not include write. These are independent grant actions that happen to share read as a common baseline.

Grant scopes

Grants can target a database (applies to all tables) or a specific table:

Scope Applies to When to use
database All current and future tables in the database Broad access for a team (for example, "traders can read everything in analytics")
table Only the named table Fine-grained control (for example, "viewers can only query the prices table")

flowchart LR

    DB["Database Grant"] -->|Covers| ALL["All current & future tables"]

    TG["Table Grant"] -->|Covers| ONE["Specific table only"]

End-to-end example

This traces how a token with tenant=quants and groups=[trader, viewer] gets authorized. (In this example, alice is the user in the IdP – but KDB.AI only sees the token's claims, not the user identity.)

1. Token issued by IdP:

Claim Value
iss https://idp.example.com/tenants/quants
aud kdbai-service
tenant quants
groups ["trader", "viewer"]

2. KDB.AI validates the token:

Check Result
Signature valid (JWKS) Pass
iss in OAUTH_ISSUERS Pass
aud matches OAUTH_CLIENT_ID Pass
Token not expired Pass
groups claim is present and not empty Pass

3. KDB.AI evaluates grants for tenant=quants, groups=[trader, viewer]:

Grant Tenant Groups Database Actions Matches token?
#1 quants trader analytics read Yes – tenant and group match
#2 quants trader analytics write Yes – tenant and group match
#3 risk viewer analytics read No – wrong tenant (risk != quants)

4. Result:

  • Grants #1 and #2 match – the token's tenant (quants) and one of its groups (trader) align with the grant. The token gets read and write access to the analytics database.
  • Grant #3 does not match – even though the token has a group called viewer, the grant is for tenant risk and the token is from tenant quants. Tenants are strictly isolated.

Full permission matrix (example setup)

Tenant Group User(s) Grants on analytics db Effective access
quants trader alice read, write Query, search, insert, create tables. Cannot delete.
quants viewer alice, bob read Query and search only. Cannot insert or delete.
risk viewer charlie read Query and search only. Completely isolated from quants grants.
manager admin root system_admin (through environment variables) Full access to everything. Can add/delete grants. Bypasses all ACL checks.

Key rules

  • Grants are never assigned to individual users. A grant applies to any token whose tenant and groups claims match.
  • A token with multiple groups gets combined access. KDB.AI checks all of the token's groups against all grants. If trader has read + write and viewer has read, a token in both groups gets read + write.
  • Cross-tenant isolation is strict. A grant for tenant quants never applies to a token from tenant risk, even if the group name is the same.
  • write implies read; delete implies read; but delete does NOT imply write.

Managing grants

Grant management is restricted to tokens that have system_admin privileges (that is, tokens whose tenant and groups claims match the ACL_SYSTEM_ADMIN_TENANT and ACL_SYSTEM_ADMIN_GROUP environment variables). Every grant endpoint requires this token in the Authorization: Bearer header.

API endpoints

Operation Method Endpoint
Add grants POST /api/v2/admin/grants
List all grants GET /api/v2/admin/grants
Get grant by ID GET /api/v2/admin/grants/{id}
Delete grant DELETE /api/v2/admin/grants/{id}

Grant parameters

The POST body is a JSON array of grant objects:

Parameter Type Required Notes
resource string Yes database, table, or admin
databaseName string Yes Target database name
table string No Target table (required when resource is table)
tenant string Yes Tenant ID (must match the tenant claim in the JWT)
groups list of strings Yes Group names from the IdP
actions list of strings Yes read, write, delete, or system_admin

KDBAI Python Client

The KDB.AI Python client supports OAuth 2.0 token management with automatic token refresh. It reads the OAuth 2.0 configuration from a YAML file whose path is provided when the session is created.

OAuth configuration file

The client will read OAuth 2.0 settings from a YAML configuration file that supports three configuration modes, depending on how tokens are obtained:

  1. Resource owner password (grant_type: password) – obtain a token for a user.

  2. Client credentials (grant_type: client_credentials) – machine-to-machine authentication.

  3. External token (grant_type: external_token) – tokens are acquired externally and supplied to the client.

Create a YAML configuration file containing the OAuth 2.0 details required by the client. The fields you need to populate depend on the grant type you use:

# --- Required ---
token_url: "<your-idp-token-endpoint>"
client_id: "<your-client-id>"
grant_type: "password"  # or "client_credentials" or "external_token"

# --- Tokens ---
# external_token: populate these values with tokens from your login script before connecting.
# password / client_credentials: the client acquires and updates these automatically. Your IdP may only return an access_token — this is fine.
access_token: "<your-access-token>"
refresh_token: "<your-refresh-token>"

# --- Only required if using client_credentials grant ---
client_secret: ""

# --- Only required if using password grant ---
username: ""
password: ""

# --- Optional: OAuth 2.0 scopes (required by some IdPs, e.g. Microsoft Entra ID) ---
scope: ""

OAuth 2.0 Configuration Fields

Field Required Description
token_url Yes Your IdP's token endpoint. Required for the client to acquire or refresh tokens.
client_id Yes The OAuth 2.0 client ID registered in your IdP
grant_type Yes The OAuth 2.0 grant type to use: password, client_credentials, or external_token
access_token Yes (for external_token) Current access token. For password and client_credentials, the client acquires this automatically.
refresh_token No Current refresh token. Enables the client to obtain new access tokens automatically.
client_secret No Client secret for confidential clients. Enables the client_credentials grant.
username No Username for password grant
password No Password for password grant
scope No Space-separated OAuth 2.0 scopes. Required by some IdPs (e.g. Microsoft Entra ID). Must include offline_access to receive a refresh token from Entra ID.

Token endpoint URL by IdP

  • Keycloak: https://<keycloak-host>/realms/<realm>/protocol/openid-connect/token
  • Entra ID: https://login.microsoftonline.com/<tenant-id>/oauth2/v2.0/token
  • Okta: https://<your-domain>.okta.com/oauth2/<authorization-server-id>/v1/token

Microsoft Entra ID requires scope

Entra ID will reject token requests that do not include a scope parameter. Additionally, offline_access must be included in the scope to receive a refresh_token — omitting it will cause a KeyError: 'refresh_token' at runtime.

scope: "api://<client-id>/access offline_access"

Connect using the Python client

Replace "oauth_config.yaml" with the path to your config file. See the OAuth configuration file example above.

import kdbai_client as kdbai

session = kdbai.Session(
    host="localhost", port=8082, mode='qipc',
    oauth={'enabled': True, 'config_file': "oauth_config.yaml"}
)
print(session.version())

Recommendation: use TLS in production

Encrypt connections to protect OAuth tokens in transit. KDB.AI does not terminate TLS directly — you need a TLS-terminating proxy (for example, nginx or a load balancer) in front of it. The proxy handles encryption and forwards connections to KDB.AI unencrypted. Add options={'tls': True} to enable TLS on the client side:

session = kdbai.Session(
    host="localhost", port=8082, mode='qipc',
    options={'tls': True},
    oauth={'enabled': True, 'config_file': "oauth_config.yaml"}
)
import kdbai_client as kdbai

session = kdbai.Session(
    endpoint="http://localhost:8081", mode='rest',
    oauth={'enabled': True, 'config_file': "oauth_config.yaml"}
)
print(session.version())

Recommendation: use TLS in production

Encrypt connections to protect OAuth tokens in transit. KDB.AI does not terminate TLS directly — you need a TLS-terminating proxy (for example, nginx or a load balancer) in front of it. The proxy handles encryption and forwards connections to KDB.AI unencrypted. Use https:// in the endpoint URL to connect over TLS:

session = kdbai.Session(
    endpoint="https://localhost:8081", mode='rest',
    oauth={'enabled': True, 'config_file': "oauth_config.yaml"}
)

Initial token acquisition

How tokens are obtained depends on the grant type:

Method Grant type Notes
Client credentials client_credentials Requires a confidential client (client_secret must be set). Tokens typically do not include a refresh token – the client re-acquires when expired.
Resource owner password password Requires username and password.
External token external_token Tokens are acquired externally by the user and written to the config file. The client reads and refreshes them from there.

For external_token, obtaining the initial access_token and refresh_token is the responsibility of the end user. A common approach is a dedicated login script that performs an OAuth 2.0 flow against your IdP – for example, device code or PKCE – and writes the resulting tokens to the YAML config file. Which flow to use may depend on how your IdP is configured and your security requirements. The KDB.AI client then reads these tokens and handles refresh from that point forward.

Token refresh

Once tokens are in the config file, the Python client handles the token lifecycle automatically:

  1. The client sends a request to KDB.AI with the current access_token
  2. If the server rejects the token (expired or invalid), this triggers the refresh flow
  3. The client attempts to obtain a new token based on the grant_type:
    • For password and client_credentials: the client re-acquires a fresh token from the IdP
    • For external_token: the client uses the refresh_token to obtain a new access_token. If the refresh_token has also expired, the client errors and the user must re-run their login script to obtain new tokens
  4. The updated tokens are written back to the config file

external_token: refresh token expiry

Once the refresh_token expires, the client cannot recover automatically. You must re-run your own login script to obtain a new access_token and refresh_token. How long a refresh token remains valid depends on your IdP configuration.

Troubleshooting

The issues below apply to all identity providers. For IdP-specific troubleshooting, refer to the guide for your IdP

KDB.AI access errors

Check the KDB.AI container logs for the exact rejection reason:

Decoding a token

Several fixes below require inspecting the token's claims. Replace $TOKEN with the variable holding your token:

echo $TOKEN | cut -d. -f2 | base64 -d 2>/dev/null | jq .
Log message Cause Fix
Invalid issuer in token: <url> The token issuer is not in OAUTH_ISSUERS Decode the token and copy the exact iss value. Add it to OAUTH_ISSUERS — it must match exactly, including any trailing slash
Invalid aud in token Token audience does not match OAUTH_CLIENT_ID Decode the token and check the aud field. It must match OAUTH_CLIENT_ID. For service principals use scope=api://<client-id>/.default; for user accounts use scope=api://<client-id>/access
Token has expired Token lifetime has elapsed Re-acquire the token and retry
groups can not be empty in token The claim named in OAUTH_GROUPS_CLAIM is present but empty Decode the token and check the claim. If empty, the identity has no group memberships — add it to the appropriate group and re-acquire the token. If the claim name is wrong, update OAUTH_GROUPS_CLAIM to match the claim in your token.
Missing field in token: groups The claim named in OAUTH_GROUPS_CLAIM is not present in the token Decode the token and confirm the claim is absent. If missing, configure your IdP to include it. If the claim name differs, update OAUTH_GROUPS_CLAIM to match.
Token signature verification failed KDB.AI could not verify the token signature Confirm KDB.AI can reach the IdP's JWKS endpoint; check network/firewall rules
requires admin privilege A non-admin identity attempted a grant or admin operation Decode the token and confirm it contains the group matching ACL_SYSTEM_ADMIN_GROUP

SSL certificate errors

If KDB.AI cannot fetch JWKS keys from your IdP, you may see an error similar to:

SSL certificate problem: self-signed certificate in certificate chain

This is typically caused by a corporate root CA certificate not being trusted inside the kdbai-db container. Contact your platform or network team to have the appropriate CA certificate mounted into the container's trust store.

JWKS endpoint unreachable

KDB.AI fetches signing keys from the issuer URL configured in OAUTH_ISSUERS. That URL must be reachable from the kdbai-db container at runtime. If it is blocked by a firewall or network policy, token validation will fail.

KDB.AI requires outbound HTTPS (TCP 443) access to two endpoints per issuer. The exact URLs are derived from OAUTH_ISSUERS:

Step Endpoint Purpose
1 <issuer>/.well-known/openid-configuration OpenID Connect discovery — KDB.AI fetches this to locate the jwks_uri for the configured issuer
2 JWKS URI returned in the discovery document Public key retrieval — KDB.AI fetches the RSA public keys used to verify JWT signatures; keys are cached and refreshed every 2 hours

If submitting a firewall request, include both endpoints. For IdP-specific endpoint URLs, refer to your IdP's documentation or the setup guide for your IdP.