Refinery watchlist management

A 'watchlist' is the term used to refer to the instrument set that a feed handler is subscribed to (watching). As instrument identifiers change over time, tools are needed to manage these watchlists and ensure that the data being subscribed to is correct.

The Watchlist Management dashboard consists of 4 tabs:

  1. Watchlist Status: This tab contains the status of the subscriptions.

  2. Watchlist Editor: This tab allows you to edit existing subscriptions and add new subscriptions.

  3. Watchlist Uploader: This tab allows you to upload a completely new watchlist without needing to edit every row manually in the Watchlist Editor

  4. Tick Loss Checks: This tab contains information about missing ticks.

Watchlist status

When the feed handler tries to make a subscription to the feed for a particular instrument, it receives a response back that contains information about the status of the subscription.

The monitoring real-time engine (daas\_mon\_rte\_a) subscribes to all of these responses from the feed and stores them in a table in memory called .daas.mon.respStatus. This table is displayed in the Watchlist Status tab, grouped by the statusCode of the response. Color coding rules have been set up to make it easier to spot subscriptions that need attention.

You can remove the automatic grouping by clicking the x beside statusCode above the data grid.

Each column in this table can be searched by typing in the white search boxes; for example, if you want to know the status of a subscription for VOD.L you can type VOD.L in the search box below the sym column heading.

Below is a table of the types of messages that might appear in this dashboard.

In practice, you generally only see:

  • Status Code: None, Not Found, Not Authorized

  • Data State: Ok, Suspect

  • Stream State: Open, Closed

Watchlist editor

On the left-hand side of this dashboard, you can see the list of your watchlists split up by asset class and market level.

If you select a particular watchlist, the table on the right will update with the list of instruments that are in that watchlist. Note that all of the watchlists use the same schema. Level 1 data is subscribed to using the market-price column and Level 2 (aka Market Depth) data is subscribed to using the market-by-price column.

You can edit a particular RIC by double-clicking on it. In the below image we are editing RB.L:

Once the changes have been made, click Enter:

You can save your changes by clicking Submit Changes in the toolbar at the top of the screen:

If you want to add a new subscription:

  1. Click Edit, the buttons will change to those visible in following figure:

  2. Enter the new RIC in the row at the bottom on the table.

  3. Click Submit Changes.

A row can be deleted by selecting it and clicking Delete 'x'. Rolling back to the original state is made by clicking Cancel Editing. You can also export the watchlist by using CSV.

Watchlist uploader

If you want to upload a completely new watchlist, you can do so using the Watchlist Uploader tab. This tab allows you to upload a CSV file containing the instruments that you want to subscribe to.

Format of the CSV

The format for this CSV is shown below in Excel:

The postfix of the filename must match one of the feed handler names, depending on the asset class and market level that you want to update

  • * emea_rfavelocity_tr_marketData_eq_0.csv

  • * emea_rfavelocity_tr_marketData_eqL2_0.csv

  • * emea_rfavelocity_tr_marketData_eqL2Legacy_0.csv

  • * emea_rfavelocity_tr_marketData_fi_0.csv

  • * emea_rfavelocity_tr_marketData_fut_0.csv

  • * emea_rfavelocity_tr_marketData_futL2_0.csv

  • * emea_rfavelocity_tr_marketData_futL2Legacy_0.csv

  • * emea_rfavelocity_tr_marketData_fx_0.csv

  • * emea_rfavelocity_tr_marketData_idx_0.csv

  • * emea_rfavelocity_tr_marketData_mn_0.csv

  • * emea_rfavelocity_tr_marketData_lisOpt_0.csv

Ensure that you have enter your instrument in the correct column

  • market-price = Level 1

  • market-by-price = Level 2

Uploading the CSV

  1. To upload a file, select the Upload button.

  2. A file explorer window will appear allowing you to select the file for upload.

    a. The new upload Watch List Config names postfix must be a *.csv and match the name of the associated feed handler.

For example

`{MySampleList_emea_tr_eq_rfavelocity.csv}`

  1. The upload file must have the extension .csv

    a. If the file is renamed with a date-time extension added to the end, then the upload has successfully executed the config update.

  2. If it does not have the correct file extension, it will be rejected.

    a. You can remove the upload that was rejected by selecting the red x icon within the table.

  3. You can download the file that you uploaded by selecting the download icon.

  4. Once the file is uploaded, you can go back to the config screen to check the config has been updated successfully.

Custom symbology mapping

The custom symbology mapping functionality enables you to query data using alternative symbol identifiers not stored in the database. For example, you want to query the symbol `VOD.L but you want to request it as `voda. To enable this, the system requires that the reference data be maintained. A CSV of the configuration must be dropped in the filestore, named to match the regex "\*SymbologyMap.csv". Formatted as below:

Column header Description Example
inputSymbol The symbol that you will now query with `test
mappedSymbol The symbol or symbols that will be extracted in the query `$"AT0A0U299=TWEB"
alwaysMap Boolean flag, map the symbol regardless of whether the user has specified in the query to enable symbology mapping 1
returnAsMappedSym If true, return the symbol as it appears on disk. If false, return as queried. 0

In the example above, querying with `test will now query AT0A0U299=TWEB regardless of user-specified flag and return the result set with the symbol `test.

This functionality can be used to create custom symbol groups. For example, you could provide an inputSymbol of `mySymbols, a mappedSymbol of `VOD.L`BARC.L`IBM.L, and set returnAsMappedSym to 1. This would enable you to query using `mySymbols and be returned with multiple symbols.

The format of the mappedSymbol field must be that of a kdb symbol list as shown above; symbol identifiers with special characters (e.g. = #, etc) must be symbol cast from string. For example:

`$"AT0A0U298=TWEB";`$"AT0A0U299=TWEB"

This CSV is uploaded to the reference data section in client file upload on the dashboards.

Note

Dropping a new symbology map here will overwrite the current one, so the CSV should be maintained and updated.

Tick loss checks

These tests run in real-time and are designed to look for any data that has been dropped from the feed.

Note how the tests run (see details below) and whether they work or not (and hence should be enabled or not) is very data dependent. If your content does not match the description provided below, these tests will not work.

The tick loss checks tab contains two sections.

  1. Tick Loss Summary

  2. Tick Loss Details

The check is done by comparing the tickCount of an incoming trade tick for a particular instrument to the previous tickCount that came in for that same instrument. In practice, the tickCount should increment by 1 for each new tick that comes in (ignoring some details such as pre-day trades, etc.):

When the difference between an incoming tickCount and the previous tickCount is not 1, this is flagged in the Tick Loss Checks tab. The Tick Loss Summary section contains a table of instruments that have been flagged for lost ticks. This table contains the time of the tick that caused this instrument to be flagged, the table name and the sym:

When you click on a particular row in this table, the Tick Loss Details section will update with the two ticks that caused the error to be flagged (the incoming tick and the tick that it was compared to):