Skip to content

REST server

Expose a RESTful interface to a kdb+ based system

We load the REST server library and create a simple API and async query server.

Example customers API

Create a new directory and copy into it rest-server.qpk.

Create in it qp.json:

{
  "customers": {
    "entry": [ "customers.q" ],
    "depends" : ["rest-server"]
  }
}

Create customers.q from the source in the appendix.

Build and run.

qp build
qp run customers

For a full overview of the server and the concepts introduced see the customer example server.

Async query server

A REST server to run asynchronous query jobs

Jobs are run in worker processes that execute arbitrary qSQL and trigger a callback when done. (A REST client is expected to poll for async job results.)

Create a new directory and copy rest-server.qpk into it

Create qp.json:

{
  "server": {
    "entry": [ "queryserver.q" ],
    "depends" : ["rest-server"]
  },
  "client": {
    "entry": [ "queryclient.q" ]
  }
}

Copy the linked examples to create queryserver.q, queryclient.q, and queryworker.q.

Build.

qp build

Create qpbuild/docker-compose.yml:

version: "3.7"
networks:
  kx:
    name: kx
    driver: bridge
services:
  server:
    image: "${server}"
    ports:
      - 8080:8080
    volumes:
      - /home/${USER}/.qp.licenses:/opt/kx/lic
    env_file:
      - qpbuild/.env
    command: -p 8080
    networks:
      - kx
    tty: true
    stdin_open: true
  client:
    image: "${client}"
    volumes:
      - /home/${USER}/.qp.licenses:/opt/kx/lic
    env_file:
      - qpbuild/.env
    command: -server http://server:8080
    networks:
      - kx
    tty: true
    stdin_open: true

It starts a REST server example and client. It forwards the server’s port 8080 so, besides the client being able to access it, you can access it outside the Docker environment.

Run.

docker-compose up

Protecting a backend API

We can provision a protected public gateway to allow access to the private HTTP backend.

With the customers API example, you should have your server exposed on port 8080.

The guides below will look for an HTTP backend endpoint, this is the host and port of the Customers example.

To follow along, you should run the HTTP customers example on a VM instance and expose port 8080. We will then use a gateway to secure access to the REST server running on 8080.

In pratice you will want 8080 exposed to the virtual network the gateway is on, but not externally to the wider web.

Your HTTP backend and Gateway should also consider requiring custom headers of your own design to quickly ignore foreign requests. This is not covered by the guides below.

Amazon API Gateway

Amazon API Gateway can be used to secure a REST API backend.

Startup a VM instance and run the customers example on port 8080 and expose port 8080 on the VM's network.

We will now use Amazon API Gateway to secure access to the customers API backend by importing an API configuration.

Setting up Amazon API Gateway

A sample OpenAPI 2.0 configuration for the customers API is included from the appendix.

After importing the API configuration, you will need to create a stage. You can may name the stage kxce-stage.

The kxce-stage should define a single stage variable httpUrl, and the value should be the URL of the customers backend. When entering this variable enter the URL without the protocol prefix. For example set httpUrl to myserver.com, not http://myserver.com.

After deploying the stage, Your API will now be accessible over HTTPS using the Stage URL, however it will not require authentication.

To require authentication to use an API, you will want to continue to read about the options Amazon provides.

Amazon API Gateway Authentication

To use IAM for authentication, you may attach an IAM Autenticatior under Develop > Routes.

Azure API Management

Azure API Management service instances can be used to create a gateway to allow access to one or more REST endpoints.

Azure API Management Key Concepts

To protect your API backend, we recommend using Azure Active Directory as the OAuth2 provider when using the API Management Service.

If you do not already have an Azure Active Directory tenant, and an Azure API Management service instance running, create new ones. This will take a while. The name of your Azure API Management service instance will become part of the gateway URL.

Startup a VM instance and run the customers example on port 8080 and expose port 8080 on the VM's network.

We will use Azure API Management to secure access to the customers API backend.

Once the Azure API Management Instance is up, import the sample OpenAPI JSON from the appendix.

After importing the API, click on the API's Design tab, and enter the Customer examples HTTP backend endpoint URL for all operations.

With this alone, you should now be able to access your REST server from the API Management gateway URL, which at the time of writing is shown in the API Management Overview.

The APIs are still unauthenticated at this point, however the Gateway should be functional.

Once you have an Azure AD tenant running, you will want to go to App Registrations and follow the instructions here for making an API and a client app.

Make sure to follow the last step, without that step OAuth2 will be confusingly enabled, yet not required. At the time of writing, that last step mentions editing a policy in raw XML. You may also edit the policy visually in the current UI, under Inbound Processing, in the Design tab, add adding a validate-jwt type policy.

After you have the backend protected as described in the Azure walkthrough, you may want to replace the step where you treat the Azure API Portal as the app, and instead using something like kurl as the client app.

Google Cloud API Gateway

Google API Gateway can be used to secure a REST API backend.

Startup a VM instance and run the customers example on port 8080 and expose port 8080 on the VM's network.

We will now use Google API Gateway to secure access to the customers API backend by importing a configuration.

Setting up Google API Gateway

Installing Google API Gateway in the Console

If you have not done so before, opening Google API Gateway in the Cloud Console will have you "install" and accept a user agreement.

Once you have Google API Gateway opened in the Google Console, you may import an API config.

When asked to import a configuration, you may use the sample OpenAPI 2.0 configuration for the customers API. Take care to replace the x-google-backend address with the address of your VM running the REST server demo.

When asked to create a Gateway, you may name it kxce-gateway, and select a region of your chosing.

Once Google finishes uploading your API, you may now make a request to https://GATEWAY_URL/customers to use the customers API, where GATEWAY_URL is the URL listed in the Google Console's Gateway tab.

For example, the generated gateway URL would look like:

https://kxce-gateway-5uoiifib.ue.gateway.dev/customers

Your API will now be accessible over HTTPS using the Gateway, however is not yet secured.

To secure the API, you may will want to continue to read about the options Google provides.

For simplicity, and to finish off with this demo, follow the steps for securing with an API Key.

For further authentication options instead of using API keys, see Authenticating with a Service Account.