Calendar data

Schema

The file name must conform to the regex *Calendar.csv and contain data which can be mapped to the following schema.

Column name	Datatype	Description	Example/s
MIC	symbol	ISOMIC of some venue	XLON
event	string	Name of an event such as a trading phase or holiday.	tradeOpen pmTradeClose Christmas Day
validFrom	date	Earliest effective date of the event.	05/07/2018m 2005.09.13
dayOfWeek	string	Pipe-delimited list of characters identifying weekdays for which the event occurs using first 3 letters of the day name. May be omitted for holidays.	mon|tue|wed|thu|fri|sat|sun
time	time	Time of the event. May be omitted for holidays. If the holiday is a half-day, should indicate updated closing time.	07:50:00 14:00:00
holiday	boolean	Whether the event is a holiday.	1 0
status	symbol	Status of the venue during the event; only accepts open or closed.	open closed

The upload may be completed using the Client Upload dashboard. The data will be stored in the secMaster HDB in the table calendarRef. Column names in the original file may assume any values but will be as described in the table once ingested into Refinery.

Mapping schema

If the uploaded file’s schema does not exactly match the above schema, then adjust two configs to complete the mapping:

• .daas.cfg.FATVAR:{Calendar.csv}

• .daas.cfg.fwTargetDB:{Calendar.csv}

The former contains a dictionary of the datatypes of each column of the input file. The latter, a table of qSQL clauses which map the input data to Refinery’s calendarRef schema.

Default values for .daas.cfg.FATVAR:{Calendar.csv}

[

Default values for .daas.cfg.fwTargetDB:{Calendar.csv}

If the columns of the input CSV match the columns of calendarRef, then these parameters can be left as they are and the upload successfully writes the data to a kdb+ table in the secMaster process. The CSV shown in Calendar data example is an example of this.

Otherwise, the FATVAR parameter must contain both datatypes conforming to the incoming file and column names such that columns containing data to be mapped to calendarRef are named as described in the above schema.

Similarly, the fwTargetDB parameter must contain a mapping of the incoming data to calendarRef. This mapping is achieved with [qSQL statements]code.kx.com/q/basics/qsql/). The fwTargetDB parameter allows for multiple filters and aggregates to be run on the input data with expressions entered into the whereClause and updateCols columns, respectively. The data in the following figure is an example of a client-specific CSV that requires additional mapping.

Example of data not conforming to Refinery calendar schema:

FATVAR values adjusted to accept example client-specific data:

FATVAR values adjusted to accept example client-specific data:

[

Invalid data

There is a special case that can cause the upload of calendar data to Refinery to fail. In this case, it is found that the earliest valid entries for a venue’s open and close times have different effective dates. For example, the oldest effective date for a tradeOpen event occurs a few days before the oldest effective date for a tradeClose event as in the following example:

If these two records constitute the first record for these events (i.e. no other trade Open/Close event has a validFrom parameter earlier than these dates), then the upload process with fail. However, asymmetric updates to open/close events are acceptable and will pass through the upload process unimpeded.