Skip to content

Use HTTP with q

This page explains how to serve HTTP requests from a q process, customize and secure the request handlers, and send HTTP requests from q to other servers.

q has a built-in web server. A q process that listens on a port answers HTTP requests on the same port that serves q IPC and WebSocket connections. A browser, curl, or any HTTP client can query it without extra software. q can also act as an HTTP client and send requests to other servers.

Serve HTTP requests

To turn a q process into a web server, listen on a port:

q)\p 8080

The process accepts HTTP requests on port 8080. For HTTPS, first configure the process for SSL/TLS.

Use the default GET handler

Without any configuration, q answers GET requests with its default .z.ph handler. Visit http://localhost:8080 in a browser on the same machine to browse the variables and views in the session.

The handler evaluates a q expression in the query string and returns the result as an HTML page. For example, http://localhost:8080/?1+1 returns a page that shows 2.

Prefix the query with a file extension to choose the response format. Given a table t, each of these requests returns it in a different format:

Request Response format
http://localhost:8080/.csv?t CSV
http://localhost:8080/.txt?t Tab-separated text
http://localhost:8080/.json?t JSON, one object per row

For example, the CSV request returns:

sym,px
a,1
b,2
c,3

The default handler runs arbitrary code from any client

Any client that can reach the port can run any q expression, including expressions that change or delete data. Before you expose a port, authenticate requests and replace the default handler with custom handlers that accept only the requests you expect. See also Permissions with q for ways to restrict HTTP queries.

Serve static files

Set .h.HOME to a directory to serve the files in it. Its default value is "html", a directory relative to where q started.

For example, with an index.html file in the directory /webserver:

q).h.HOME: "/webserver"

Then the page is available at http://localhost:8080/index.html.

For a full example of customizing the default web server, see the doth example web server on GitHub.

Write custom handlers

Define a callback in the .z namespace for each HTTP method that the server supports:

Callback HTTP methods Argument
.z.ph GET (requestText; headers)
.z.pp POST (requestText; headers)
.z.pm OPTIONS, PATCH, PUT, DELETE (method; requestText; headers)

In each argument, headers is a dictionary of the HTTP request headers, with header names as symbols. requestText is the request path without its leading /. For a request with a body, such as a POST, requestText holds the path, a space, and then the body.

Each callback returns a complete HTTP response as a string. The .h namespace builds the common ones: .h.hy wraps content in a 200 OK response of a given type, and .h.hn builds a response with any status.

The script below serves a small table of prices as JSON. It answers GET, POST, and DELETE requests, and rejects other methods with a 405 status:

/ prices.q
\p 8080
prices: ([sym: `AAPL`MSFT] px: 227.5 415.2)

/ GET /AAPL returns the price row for AAPL as JSON
.z.ph: {[x] .h.hy[`json] .j.j 0!select from prices where sym = `$x 0}

/ POST / with a JSON body upserts a price
.z.pp: {[x]
  body: (1 + x[0]?" ") _ x 0;          / drop the path and the separating space
  d: .j.k body;
  `prices upsert (`$d`sym; d`px);
  .h.hy[`json] .j.j count prices
  }

/ DELETE /MSFT removes a row; other methods get a 405
.z.pm: {[x]
  $[`DELETE = x 0;
    [delete from `prices where sym = `$x 1; .h.hy[`json] .j.j count prices];
    .h.hn["405 Method Not Allowed"; `txt; "unsupported method\n"]]
  }

Start the server with q prices.q, then send requests to it from a shell:

$ curl http://localhost:8080/AAPL
[{"sym":"AAPL","px":227.5}]
$ curl -d '{"sym":"NVDA","px":181.3}' http://localhost:8080/
3
$ curl -X DELETE http://localhost:8080/MSFT
2
$ curl -X PUT http://localhost:8080/MSFT
unsupported method

.j.j serializes the q results to JSON, and .j.k parses the JSON body of the POST request.

Customize the default handler instead

The default .z.ph evaluates expressions through .h.val. To keep the default handler's formatting and browsing but control what it runs, redefine .h.val rather than .z.ph.

Authenticate requests

Define .z.ac to authenticate and authorize every HTTP request before q passes it to a handler. Like .z.ph, it receives (requestText; headers), so it can inspect tokens, cookies, or any other header. This lets you secure the server with mechanisms such as Lightweight Directory Access Protocol (LDAP), OAuth 2.0, and OpenID Connect.

.z.ac returns a two-item list that tells q what to do with the request:

Return value Effect
(0; "") Reject the request with a 401 Unauthorized response
(1; "username") Accept the request, set .z.u to the username, and run the handler
(2; "response text") Reject the request with your own HTTP response
(4; "") Fall back to basic access authentication

Without .z.ac, q uses basic access authentication, which checks the username and password against the -u or -U password file and .z.pw, if they are set.

For example, add this line to prices.q to accept only requests that carry a known bearer token:

.z.ac: {[x] $["Bearer s3cret" ~ x[1]`Authorization; (1; "svc"); (0; "")]}

Then requests without the token get a 401 response, and the handlers never run:

$ curl -i http://localhost:8080/AAPL
HTTP/1.1 401 Unauthorized
Connection: close
Content-Length: 0
WWW-Authenticate: Basic realm=""

$ curl -H "Authorization: Bearer s3cret" http://localhost:8080/AAPL
[{"sym":"AAPL","px":227.5}]

Don't hard-code credentials

The token in this example only shows the mechanism. In production, validate tokens against an identity provider, and serve the process over HTTPS so that credentials are encrypted in transit.

Keep connections alive

By default, q closes the connection after each response. To let a client reuse a connection for further requests, call .h.ka inside a handler. It takes an idle timeout in milliseconds as an integer and returns the value for the Connection response header: "keep-alive", or "close" if the timeout is 0i.

/ build a text response that keeps the connection open for up to ka milliseconds
resp: {[ka; body] "HTTP/1.1 200 OK\r\nConnection: ", .h.ka[ka], "\r\nContent-Type: ", .h.ty[`txt], "\r\nContent-Length: ", string[count body], "\r\n\r\n", body}
.z.ph: {[x] resp[2000i; "pong\n"]}

Send two requests in one curl command, and its verbose output shows the second request reusing the first connection:

$ curl -v http://localhost:8080/ping http://localhost:8080/ping
...
* Re-using existing connection with host localhost
...

Compress responses

Since v4.0 2020.03.17, the default handler compresses responses with gzip when both of these are true:

  • The client sends the Accept-Encoding: gzip request header.
  • The response has at least 2,000 characters.

The response then carries the Content-Encoding: gzip header. Compression applies to query requests such as http://localhost:8080/.csv?t, and needs no configuration.

Send HTTP requests

q can send HTTP requests to other servers and to other q processes. The examples in this section send requests to the prices.q server from Write custom handlers, running on the same machine without the .z.ac authentication.

Send GET and POST requests

.Q.hg sends a GET request to a URL and returns the response body as a string. Pass any query parameters in the URL:

q).Q.hg "http://localhost:8080/AAPL"
"[{\"sym\":\"AAPL\",\"px\":227.5}]"
q).j.k .Q.hg "http://localhost:8080/AAPL"   / parse the JSON response
sym    px
------------
"AAPL" 227.5

.Q.hp sends a POST request. It takes the URL, the MIME type of the body, and the body. .h.ty maps a file type to its MIME type:

q).Q.hp["http://localhost:8080"; .h.ty`json] .j.j `sym`px!("NVDA"; 181.3)
,"3"

Both functions:

  • Accept a URL as a string or a symbol
  • Ask the server for gzip compression, and decompress the response before they return it
  • Use the proxy set in the http_proxy or HTTP_PROXY environment variable, and skip it for the domains in no_proxy or NO_PROXY
  • Accept a username and password in the URL for basic access authentication, for example "http://username:password@localhost:8080/AAPL"

Send raw HTTP requests

To send any other method, or to set your own request headers, apply a connection handle to a complete HTTP request string:

`:http://host:port "request text"

q returns the whole HTTP response as a string, including its status line and headers.

An HTTP request consists of:

  1. A request line: the method, the path, and the protocol version
  2. Zero or more header fields, each of the form Name: value
  3. An empty line
  4. An optional message body

Each line, including the empty one, ends with a carriage return and a line feed, "\r\n".

For example, this request deletes a row from the prices table. The Connection: close header asks the server to close the connection after the response:

q)r: `:http://localhost:8080 "DELETE /MSFT HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n"
q)r
"HTTP/1.1 200 OK\r\ncontent-type: application/json\r\nconnection: close\r\nco..

An HTTP response has the same structure as a request, with a status line in place of the request line. Split the response on the empty line to separate the headers from the body:

q)`head`body set' "\r\n\r\n" vs r;
q)"\r\n" vs head
"HTTP/1.1 200 OK"
"content-type: application/json"
"connection: close"
"content-length: 1"
q)body
,"2"

A request with a body needs a Content-Length header that gives the size of the body in bytes:

q)body: .j.j `sym`px!("TSLA"; 251.1)
q)r: `:http://localhost:8080 "POST / HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\nContent-Type: application/json\r\nContent-Length: ", string[count body], "\r\n\r\n", body
q).j.k .Q.hg "http://localhost:8080/TSLA"
sym    px
------------
"TSLA" 251.1

Raw requests are also how you send headers that .Q.hg and .Q.hp don't, such as the bearer token in Authenticate requests:

q)`:http://localhost:8080 "GET /AAPL HTTP/1.1\r\nHost: localhost\r\nAuthorization: Bearer s3cret\r\nConnection: close\r\n\r\n"

Chunked responses arrive whole

Since v3.3 2014.07.31, if a server sends its response with chunked transfer encoding, q joins the chunks before it returns the response.

Send HTTPS requests

To send a request over HTTPS, first configure the process for SSL/TLS, then use https in place of http in the URL:

q).Q.hg "https://example.com"

.Q.hg and .Q.hp can't send HTTPS through every proxy

They don't support HTTPS across a proxy that requires the CONNECT method.

Format HTTP and HTML

The .h namespace holds tools for formatting markup and HTTP responses, such as .h.hy and .h.hn for responses and .h.ty for MIME types. It also has functions that render q data as HTML, CSV, or XML.

Next steps