Skip to content

REST server

Expose a RESTful interface to a kdb+ based system

Quick start

In a nutshell, it lets you map REST operation-and-path combinations (endpoints) to q functions (handlers). When a HTTP request is received it matches it to an endpoint; when one is found it processes the request according to its definition, and executes its handler.

The operation is arbitrary, but typically one of HTTP methods defined by REST specification: GET, POST (create), PUT (update), or DELETE.

The path is the part of the URL that comes after the host and port (e.g. /customers). It identifies the resource subject of the operation. The path is arbitrary and may contain variables (which are treated as parameters to the endpoint), for example /db/{table}/schema is a valid path that has a variable named table.

Usage

To use the library

  1. Create the endpoint mappings with .rest.register
  2. On receipt of a request to .z.ph or .z.pp, forward input to .rest.process to process the incoming request and return processing result

Example:

//
// Register endpoints.
//
.rest.register[`get;"/customers";
    "Returns all customers";
    .db.getAllCustomers;
    .rest.reg.data[`i;-6h;0b;0;"Offset to first row"],
      .rest.reg.data[`cnt;-6h;0b;10;"Number of rows to return"] ]

.rest.register[`get;"/customers/{id}";
    "Returns one or more customers by their IDs";
    .db.getCustomersById;
    .rest.reg.data[`id;6h;1b;0;"One or more customer IDs"] ]

//
// Hook request processing.
//
.z.ph:.rest.process[`GET]
.z.pp:.rest.process[`POST]

Above we created the following two endpoints:

GET /customers returns all customers. It expects 2 parameters: i (integer) is the offset of first row to return, and cnt (integer) is the number of rows to return. Both parameters are optional, and have default values of 0 and 10 respectively. The endpoint is mapped to .db.getAllCustomers function which can possibly be defined as follows:

.db.getAllCustomers:{x[`arg;`cnt]#select from customers where i>=x[`arg;`i]}

The endpoint can be queried:

curl 'localhost:8080/customers'
curl 'localhost:8080/customers?i=10&cnt=10'
GET /customers/{id}

returns one or more customers given their IDs. It expects one parameter: id (integer[]) is an input parameter in the form of a path variable (whose value is determined at matching time). The endpoint is mapped to .db.getcustomersByIdfunction which can possibly by defined as follows:

.db.getCustomersById:{
  select from customers where id in x[`arg;`id] //! should be in ID order
    }

Register the endpoints

An endpoint is registered using .com_kx_rest.register function. The registration contains the following components:

  • The operation and path of the endpoint
  • A description summarizing the purpose of the endpoint
  • The handler function
  • Optional definition of user input, which includes: path variables, query string, headers, and request body (for post/put-based endpoints).
  • Optional definition of result object

The operation is arbitrary, but typically is one of the following: get (for data retrieval), post (to create one or more objects), put (to update one or more objects), or delete (to delete an object).

The path is arbitrary, and may contain variables whose values are determined at matching time (e.g. /db/{tbl}/schema). The operation and path combination uniquely identifies the endpoint.

The handler function is responsible for performing the activity of the endpoint. It is explained in Request Processing section.

User input definitions specify the properties of expected input parameters which include path-variables and query-string (using .com_kx_rest.reg.data), headers (using .com_kx_rest.reg.header), and JSON based request body (using .com_kx_rest.reg.body).

For example the request /db/{x}/{y}/z?i=0&cnt=10 has the following input parameters: x, y, i, and cnt. The developer can specify the name, data type, necessity, default value, and description of each parameter. The framework uses this information to

  • Ensure required parameters are in the request, failing the request if any is missing
  • Parse value using the expected data type

The handler also has access to raw user input, which is useful when some or all of the input is not known at registration time.

Output definition specifies the schema of the result (using .com_kx_rest.reg.output). The definition is currently not used by the framework, but it is a good practice to specify it for future purposes. For example, a future version will support generating openAPI documents based on the registrations.

Request processing

A HTTP request contains the following components

  • HTTP method
  • Path
  • Query string
  • HTTP headers
  • Request body (applies to POST, PUT, and DELETE)

The HTTP method generally specifies the REST operation, but in kdb+ it is limited to GET and POST. Thus, the framework looks for the operation in the http-method HTTP header, and if not present falls back to GET (for .z.ph), or POST (for .z.pp). It is thus important to use a front-end API gateway (e.g. AWS HTTP API gateway) that can populate http-method HTTP headers, and convert PUT, and DELETE HTTP methods to POST, leaving GET requests as they are.

When a request is received, both operation and path are combined to find a matching endpoint, favouring exact matches over ones containing variables (e.g. /a/b/c vs /a/{x}/c).

User input is then processed, distinguishing between the following categories:

  • User input specified using .com_kx_rest.reg.data function. These contain path variables (e.g. id in /customers/{id}) and query-string parameters. Values are parsed according to their data-type. If a parameter is missing from the request, then its default value is used. But if the missing parameter is marked required, then the request fails with a 400 HTTP status code , with names of missing required parameters included in the response. Values of input parameters are passed to the handler function as a dictionary under arg key. Raw input parameters (as they appear in the request) are also passed to the handler function under rawArg key.

  • Request body (where applicable) is expected to be in JSON format. It is deserialized into a kdb+ data structure by .j.k, and passed to the handler function under data key.

  • Request body (in case of post or put operation) is expected in JSON format. If the endpoint has body input specified using .com_kx_rest.reg.body function, then the elements of the defined object are parsed from the input (including nested objects) according to their datatypes and necessity settings. Similar to input parameters, the raw body is passed to the handler under rawData key.

  • HTTP headers are are passed as-is to the handler function under the hdr key.

After input is collected, the unary handler function is executed, its single argument a dictionary with

  • REST operation of the endpoint
  • The path of the endpoint
  • Values of processed input parameters
  • Raw input parameters present in the request
  • Value of processed body object, or empty dictionary if no body is specified in the endpoint registration
  • Raw KDB+ form of the request body (if present)
  • HTTP headers

The handler function is expected to return its response in one of the following forms:

  • A kdb+ data structure (typically a dictionary or a table), serialized to JSON by the framework before being returned to the client
  • Result of .com_kx_rest.util.response function, which gives the handler control over the HTTP status code, and content type of the response (available post release 1.0.0)
  • Result of the .com_kx_rest.util.httpResponse function, which gives the handler total control over the response

If there is a problem with input, the handler must call .util.throw to signal an error.


API reference