Skip to content

Get started with fsi-lib on the DB Service

This page explains how to run fsi-lib on a single-node DB Service deployment.

Prerequisites

This quickstart does not provide comprehensive DB Service installation instructions. For installation information, see the Database Service documentation.

Before you begin, ensure that you have the following:

  • DB Service. Install the latest version of DB Service and meet the prerequisites described in the DB Service documentation.
  • KX Downloads Portal bearer token. A bearer token to download packages from the KX Downloads Portal (represented by BEARER in this guide).
  • KDB-X is installed.
  • q client for DB Service. (dbservice_client.q) from the q client repo is installed into ~/.kx/mod/kx/.

Download fsi-lib

Set the fsi-lib version, as shown below. This helps simplify the download command.

export FSILIBVER=1.4.0

From the DB Service directory, download fsi-lib from the KX Downloads Portal:

curl -s --fail-with-body -D /dev/stderr --oauth2-bearer ${BEARER} -L -OJ https://portal.dl.kx.com/assets/raw/kxi-accelerators/fsi/fsi-lib/packages/${FSILIBVER}/fsi-lib-${FSILIBVER}.kxi

The .kxi file is a zip file. Unpack it into the packages/fsi-lib folder in your DB Service directory:

unzip fsi-lib-${FSILIBVER}.kxi -d packages/fsi-lib

Make the unpacked files readable by the DB Service containers. The files are unpacked with owner-only permissions, which the containers cannot read:

chmod -R a+rX packages/fsi-lib

Start the DB Service with fsi-lib

  1. Create a file named docker-compose.fsi.yaml next to docker-compose.yaml to configure DB Service to load fsi-lib:

    cat > docker-compose.fsi.yaml <<'EOF'
services:
  kx-db-da:
    volumes:
      - ./packages:/packages:ro
    environment:
      - KX_PACKAGE_PATH=/packages
      - KXI_PACKAGES=fsi-lib
  kx-db-agg:
    volumes:
      - ./packages:/packages:ro
    environment:
      - KX_PACKAGE_PATH=/packages
      - KXI_PACKAGES=fsi-lib
  kx-db-rc:
    volumes:
      - ./packages:/packages:ro
    environment:
      - KX_PACKAGE_PATH=/packages
      - KXI_PACKAGES=fsi-lib
EOF

  2. Start the service:

    docker compose -f docker-compose.yaml -f docker-compose.fsi.yaml up -d

  3. Confirm that fsi-lib loaded successfully:

    docker logs kx-db-da 2>&1 | grep "uda=.fsi.getTicks" | sed 's/.*uda=//' | sort -u

docker logs kx-db-da 2>&1 | grep "uda=.fsi.getStats" | sed 's/.*uda=//' | sort -u

    The output contains both fsi-lib APIs:

    .fsi.getStats
.fsi.getTicks

Connect to the DB Service using the q client

dbs:use`kx.dbservice_client
session:dbs.createSession[]

Create tables

This example uses the Trade and Quote market data tables. Your market data tables do not have to use these names. Use the table name in the table argument of each fsi-lib API call.

Note

Creating tables, as shown here, is how tables are set up on a single-node DB Service. In a clustered deployment the tables are defined upfront in the assembly yaml, so there is no need to create them at runtime.

To use symbology or corporate actions, you must also create the Symbology and CorpActions reference tables. Unlike the market data tables, these two tables must be named exactly Symbology and CorpActions. For instructions, see Use symbology and corporate actions with DB Service.

Create the Trade and Quote tables:

(session.createTable `table`type`prtnCol`sortColsDisk`sortColsOrd`columns!("Trade";"partitioned";"eventTimestamp";("instrumentID";"eventTimestamp");("instrumentID";"eventTimestamp");(`name`type`attrDisk!("eventTimestamp";"timestamp";"parted");`name`type`attrMem`attrDisk!("instrumentID";"symbol";"grouped";"parted");`name`type!("price";"float");`name`type!("volume";"long");`name`type!("conditions";"string");`name`type!("tradedExchange";"symbol");`name`type!("sequenceNumber";"long");`name`type!("srcSys";"symbol"))))`status
(session.createTable `table`type`prtnCol`sortColsDisk`sortColsOrd`columns!("Quote";"partitioned";"eventTimestamp";("instrumentID";"eventTimestamp");("instrumentID";"eventTimestamp");(`name`type`attrDisk!("eventTimestamp";"timestamp";"parted");`name`type`attrMem`attrDisk!("instrumentID";"symbol";"grouped";"parted");`name`type!("bidPrice";"float");`name`type!("bidSize";"long");`name`type!("askPrice";"float");`name`type!("askSize";"long");`name`type!("srcSys";"symbol"))))`status

"completed"
"completed"

Confirm that both tables exist:

session.listTables[]

"Trade"
"Quote"

Import table data

Start an import job for each of your relevant files located in data/imports/:

j1:session.importFiles`table`path!("Trade";"Trade.csv")
j2:session.importFiles`table`path!("Quote";"Quote.csv")

Check the status of your import jobs until every job is completed:

select affectedTables,status from session.getImport each (j1;j2)@\:`jobId

affectedTables status     
---------------------------
"Trade"        "completed"
"Quote"        "completed"

Confirm your data is available:

session.querySimple`table`startTS`endTS`limit!(`Quote;2020.01.01D;2030.01.01D;3)

eventTimestamp                instrumentID bidPrice bidSize askPrice askSize srcSys
------------------------------------------------------------------------------------
2020.08.27D13:30:00.000527000 AAPL         508.49   2       508.5    6       ONETICK
2020.08.27D13:30:00.001072000 AAPL         508.49   2       508.56   3       ONETICK
2020.08.27D13:30:00.001376000 AAPL         508.49   2       508.56   2       ONETICK

Connect to the fsi-lib APIs

The DB Service gateway provides access to the fsi-lib functions on port 5040. Open a connection to the gateway:

h:hopen `:localhost:5040

To call a function, provide its name and a dictionary of arguments. The gateway returns a header and the requested data. Each request requires the following keys:

Key Meaning
table Specifies the target table, such as `Trade.
idList Specifies the target securities, such as enlist `NVDA.
startTS Specifies the inclusive start of the time range.
endTS Specifies the exclusive end of the time range.

For complete API documentation, see the fsi-lib API overview.

Example: .fsi.getStats

Get the daily open, close, volume and number of trades for Apple (27 Aug 2020 to 02 Sept 2020) using .fsi.getStats:

args:(!). flip (
    (`table;`Trade);
    (`idList;enlist `AAPL);
    (`startTS;2020.08.27D);
    (`endTS;2020.09.02D);
    (`groupBy;enlist `instrumentID);
    (`agg;`open`close`volume`trades!((`first;`price);(`last;`price);(`sum;`volume);(`count;`price)));
    (`granularity;1);
    (`granularityUnit;`day)
    );

last h(`.fsi.getStats;args;`;()!())

instrumentID eventTimestamp                open   close    volume   trades
--------------------------------------------------------------------------
AAPL         2020.08.27D00:00:00.000000000 508.5  509.72   1343969  11445
AAPL         2020.08.28D00:00:00.000000000 504.1  503.3458 1308543  12608
AAPL         2020.08.31D00:00:00.000000000 127.62 127.575  14518333 69239
AAPL         2020.09.01D00:00:00.000000000 132.79 131.69   10326057 57302

Refer to getStats for the full list of agg formats, and to getTicks to retrieve the underlying records.

Back to top