REST-server library
Expose a RESTful interface to a kdb+ based system
In a nutshell, it allows the developer to map operation/path combinations (endpoints) to q functions (handlers). Then when a HTTP request is received it matches it to an endpoint, and when one is found, processes the request according to its definition, and executes its handler.
The operation is arbitrary, but typically one of HTTP methods defined by REST specification: GET
, POST
(create), PUT
(update), or DELETE
.
The path is the part of the URL that comes after the host and port (e.g. /customers
). It identifies the resource subject of the operation. The path is arbitrary and may contain variables (which are treated as parameters to the endpoint), for example /db/{table}/schema
is a valid path that has a variable named table
.
Usage
To use the library:
- Initialize the framework, using the
.com_kx_rest.init
function. - Register the endpoints, using the
.com_kx_rest.register
function. - (Optional) On receipt of a request to
.z.ph
or.z.pp
, forward execution to.com_kx_rest.process
to process the incoming request and return processing result.
To simplify the examples, we assume a .rest
namespace exists which aliases .com_kx_rest
:
/ Alias namespace for convenience, typically once at beginning of file
.rest:.com_kx_rest
.rest.init enlist[`autoBind]!enlist[1b] / Initialize
//
// Register endpoints
//
.rest.register[`get;
"/customers";
"Returns all customers";
.db.getAllCustomers;
.rest.reg.data[`i;-6h;0b;0;"Offset to first row"],
.rest.reg.data[`cnt;-6h;0b;10;"Number of rows to return"] ]
.rest.register[`get;
"/customers/{id}";
"Returns one or more customers by their IDs";
.db.getCustomersById;
.rest.reg.data[`id;6h;1b;0;"One or more customer IDs"] ]
Above we initialized the framework with autoBind
option set to 1b
, then created two endpoints.
get /customers
-
returns all customers. It expects two parameters:
i (integer)
is the offset of first row to return, andcnt (integer)
is the number of rows to return. Both parameters are optional, and have default values of0
and10
respectively. The endpoint is mapped to the.db.getAllCustomers
function, which might be defined: .db.getAllCustomers:{ x[`arg;`cnt]#select from customers where i>=x[`arg;`i] }
-
The endpoint can be queried as follows:
curl 'localhost:8080/customers'
curl 'localhost:8080/customers?i=10&cnt=10'
get /customers/{id}
-
returns one or more customers given their IDs. It expects one parameter:
id (integer[])
is an input parameter in the form of a path variable (whose value is determined at matching time). The endpoint is mapped to the.db.getcustomersById
function, which might be defined:
.db.getCustomersById:{
select from customers where id in x[`arg;`id] // should be in ID order
}
Initialize the REST server framework
The framework is initialized using .com_kx_rest.init
function. This function is overloaded such that it can be called
without arguments (e.g. .com_kx_rest.init[]
) to perform minimal initialization, or as a unary (with a dictionary argument)
to perform further steps (auto-binding to .z.ph
and .z.pp
in the above example).
When the autoBind
option is specified as 1b
, an incoming request is delegated to the next handler (if applicable) if it
cannot be matched to an endpoint. Conversely if autoBind
is not specified (or specified as 0b
), then a request that
does not match an endpoint is rejected with 404 HTTP code.
Note that when auto-binding is disabled, then it is the application’s responsibility to bind .com_kx_rest.process
to
.z.ph
and .z.pp
, as well as performing any chaining as needed.
Register the endpoints
An endpoint is registered using .com_kx_rest.register
function. The registration contains the following components:
- The operation and path of the endpoint
- A description summarizing the purpose of the endpoint
- The handler function
- Optional definition of user input, which includes: path variables, query string, headers, and request body (for post/put based endpoints).
- Optional definition of result object
The operation is arbitrary, but typically one of the following: get
(for data retrieval), post
(to create one or more objects), put
(to update one or more objects), or delete
(to delete an object).
The path is arbitrary, and may contain variables whose values are determined at matching time (e.g. /db/{tbl}/schema
). The operation and path combination uniquely identifies the endpoint.
The handler function is responsible for performing the activity of the endpoint. It is explained in the following section.
User input definitions specify the properties of expected input parameters which include path-variables and query-string (using .com_kx_rest.reg.data
), headers (using .com_kx_rest.reg.header
), and JSON-based request body (using .com_kx_rest.reg.body
). For example the request /db/{x}/{y}/z?i=0&cnt=10
has the following input parameters: x
, y
, i
, and cnt
. You can specify the name, data type, necessity, default value, and description of each parameter. The framework uses this information to:
- ensure required parameters are supplied in the request, failing the request if any is missing
- parse value using the expected data type
The handler also has access to raw user input, which is useful when some or all of the input is not known at registration time.
Output definition specifies the schema of the result (using .com_kx_rest.reg.output
). The definition is currently not used by the framework, but it is good practice to specify it for future purposes.
Request processing
A HTTP request contains the following components:
- HTTP method
- Path
- Query string
- HTTP headers
- Request body (applies to
POST
,PUT
, andDELETE
)
The HTTP method generally specifies the REST operation; in kdb+ it is limited to GET
and POST
. Thus, the framework looks for the operation in the http-method
HTTP header, and if not present defaults to GET
(for .z.ph
), or POST
(for .z.pp
). It is thus important to use a front-end API gateway (e.g. AWS HTTP API gateway) that can populate http-method
HTTP header, and convert PUT
, and DELETE
HTTP methods to POST
, leaving GET
requests as they are.
When a request is received, both operation
and path
are combined to find a matching endpoint, favouring exact matches over ones containing variables (e.g. /a/b/c
vs /a/{x}/c
).
User input is then processed, distinguishing between the following categories:
-
User input specified using
.com_kx_rest.reg.data
function. These contain path variables (e.g.id
in/customers/{id}
) and query-string parameters. Values are parsed according to their datatype. If a parameter is missing from the request, then its default value is used. But if the missing parameter is marked required, then the request fails with 400 HTTP status code (with names of missing required parameters included in the response). Values of input parameters are passed to the handler function as dictionary underarg
key. it is worth noting that raw input parameters (as they appear in the request) are also passed to the handler function underrawArg
key. -
Request body (where applicable) is expected to be in JSON format. It is deserialized into a kdb+ data structure using
.j.k
, and passed to the handler function underdata
key. -
Request body (in case of
post
orput
operation) is expected to be in JSON format. If the endpoint has body input specified using the.com_kx_rest.reg.body
function, then the elements of the defined object are parsed from the input (including nested objects) according to their datatypes and necessity settings. Similar to input parameters, the raw body is passed to the handler underrawData
key. -
HTTP headers are passed untouched to the handler function under
hdr
key.
After input is collected, the handler function is invoked. If the function is declared to take arguments with the same names as the endpoint parameters (or body
if endpoint is comprised only of a body
parameter), then the function is variadically invoked with its arguments mapped from the request input. Otherwise, the function is invoked as a unary with a dictionary containing the following keys:
- REST operation of the endpoint
- The path of the endpoint
- Values of processed input parameters
- Raw input parameters present in the request
- Value of processed body object, which is typically a dictionary if the body is of an object type, but can be of any type as specified by
reg.body
function. - Raw kdb+ form of the request body (if present)
- HTTP headers
The handler function is expected to return its response in one of the following forms:
- A kdb+ data structure (typically a dictionary or a table), which is serialized to JSON by the framework before being returned to the client
- Result of the
.com_kx_rest.util.response
function, which gives the handler control over the HTTP status code, and content type of the response (available post release 1.0.0) - Result of the
.com_kx_rest.util.httpResponse
function, which gives the handler total control over the response - If there is a problem with input, the handler must call
.util.throw
to signal an error