Gateway

Gateway is the primary access point for all Refinery API calls from Refinery 5.0.

Info

The QR/QP mechanism defers all routing and querying to the Refinery GW. Therefore, it is recommended to bypass the QR altogether if possible.

Gateway supports API queries in the following format:

(`API_function_name; `argument!`dictionary)

It is recommended to always send API calls in kdb list syntax (although strings have limited support if required).

Executing queries

Gateway supports asynchronous and deferred synchronous IPC calls. It is also recommended that you specify queryId in the API call dictionary so the result can be linked back to the query. If queryId is omitted, a random one will be generated by the Gateway on your behalf.

Synchronous execution

q)gw:hopen `:host:port

q)gw(`getTicks; `dataSource`dataType`startDate`endDate`idList!(`demo; `DemoTable; .z.d; .z.d; `))
sym time price volume
---------------------

Asynchronous execution

When running asynchronously, the calling process must have the .refinery.gw.result function defined. This function should take a single argument, which is a dictionary and returns:

• queryId: The ID of the query as a GUID
• success: Boolean if the execution was successful or not
• result: The result of the execution as a table if the query execution was successful
• error: The exception that occurred if the query execution failed

q)gw:hopen `:host:port
q).refinery.gw.result:{ -1 .Q.s x }

q)neg[gw](`getTicks; `dataSource`dataType`startDate`endDate`idList!(`demo; `DemoTable; .z.d; .z.d; `))
queryId| 409031f3-b19c-6770-ee84-6e9369c98697
success| 1b
result | +`sym`time`price`volume!(`symbol$();`timestamp$();`float$();`long$())
error  | ""

Sending API calls as strings

Info

Strings are fully supported as of version 5.11.0.

q)gw:hopen `:host:port

q)gw"getTicks `dataSource`dataType`startDate`endDate`idList!(`demo; `DemoTable; .z.d; .z.d; `)"
sym time price volume
---------------------

Accessing Refinery via dashboards

To be able to query Refinery processes from a Dashboard, you must run at least one Refinery GW client process:

refinery service-class --start-template refinery-gw-client

Once you have a Refinery GW Client process running, all Dashboard queries should be written to target refinery_gw_client and then use one of the following two modes to query data:

1. API call
2. Direct function call

API call

Use .gwClient.query.sync to execute an API call:

.gwClient.query.sync[`getTicks; `dataSource`dataType`startDate`endDate`idList!(`demo; `DemoTable; .z.d; .z.d; `abc`def)]

This call will randomly pick any of the available gateways in the system per query and execute the query on that GW.

Direct function call

Use .dashe.execute to target a specific process:

/ Target the primary Monitoring RDB and run a pre-defined function
.dashe.execute[`Monitoring.0.rdb.0; (`.daas.cmon.qgetQRQueryLog; ::)]

Adding functions to a GW client

As the GW client processes are not managed by the Process Manager, there is no YAML configuration available to add additional libraries into these processes at the moment.

For additional functionality in these processes (e.g. wrapping Refinery API calls), they should be added to dashe.q or dashe.*.q files in the domain or client package to have them loaded when the process starts up

Customizing GW client targetting

The GW client process accepts the following additional parameters to modify which Gateway should be queried (via .gwClient.query.sync):

• --instance-priority *num*: Use Gateways within the specified pipeline instance (in any pipeline) as the primary
• --pipeline-priority *name*: Use Gateways within the specified pipeline name (in any instance) as the primary

In both cases, if there is no matching Gateway based on the specified command line arguments, the GW client will fall back to querying any other available Gateway. If no Gateways are available, a NoGatewaysAvailableException will be returned.

Exceptions

Gateway will return an exception to the caller if the execution of a query fails for any reason.

• Synchronous mode: Exception is reported as a standard kdb+ exception
• Asynchronous mode: The success key will be false and the error key will be populated with the error

Runtime exceptions

All exceptions returned by the gateway will have one of the following as their prefix with more detail optionally provided.

Exception type	Description
InvalidGwFunctionException	The specified API function call to execute is null or non-symbol
GwInvalidArgumentTypeException	The argument for the API is not a dictionary
GwNoArgumentsException	The argument dictionary has no values in it
GwPreProcessingFailedException	Argument validation has failed
GwNoRouteException	The specified data hierarchy and data type fields have resulted in no target processes to execute the query

Pre-processing additional exception information

If a GwPreProcessingFailedException exception occurs, an additional exception string may be provided to provide more information.

Pre-processing exception type	Description
NoDataHierarchyException	If no data hierarchy parameters specified and allow null hierarchy configuration is not enabled
InvalidDataHierarchyException	If any of the data hierarchy parameters are null or not a symbol
MissingRequiredArgumentsException	If any required arguments are missing (see .gw.cfg.requiredArguments)
InvalidRequiredArgumentsException	If any required arguments are the wrong type (see .gw.cfg.requiredArguments)
InvalidDateArgumentsException	If the endDate parameter is before the startDate

Other exceptions

There are a number of exceptions that should only occur during customization of Refinery.

Exception Type	Description
GwInvalidPreProcessingException	The pre-processing functions returned an incorrect object to continue GW execution. Must be a dictionary
GwRoutingFailedException	The routing functions failed to execute
GwInvalidRouteException	The routing functions returned an incorrect object to continue GW execution. Must be a table
GwDownstreamExceptionException	The API execution on the underlying processes failed