Equity Analytics Framework - Util Functions
The Equity Analytics Framework calculates a wide number of Order Analytics. This framework is built with flexibility in mind, and can easily be extended with custom analytics.
A number of utility functions have been written to enable development of custom analytics.
.eqea.util.simpleAnalytics
The Util function enables users to run "simple" analytic, that is, analytics that make use of columns already present in the OrderAnalytics
table.
The function .eqea.util.simpleAnalytics
runs on a subset of the .eqea.analytics.cfg
.
Analytics that make use of .eqea.util.simpleAnalytics
should have at a minimum the following values populated in .eqea.analytics.cfg
:
- analytic
- analyticType
- funcName
- aggClause
Parameters:
Name | Type | Description |
---|---|---|
OrderAnalyticsRes | table | Table containing all OrderAnalytics data - contains the results of all analytics preceeding the current analytic type. |
cfg | table | Subset of .eqea.analytics.cfg - contains all details required for current batch of analytics. |
Returns:
Type | Description |
---|---|
table | Table containing all Analytic Results. |
Example usage:
Writing example custom function .example.orderExecution.simpleCustomAgg
:
- Populating required values in
.eqea.analytics.cfg
:
.eqea.analytics.cfg:flip `analytic`analyticType`funcName`aggClause`marketDataTabName`joinTimeOffset! flip (
// Examples of adding simple custom analytic types (simple aggregation on already calculated values in our OrderAnalytics table)
(`strikeToCompletionBidMidPrice ;`simpleCustomAgg ;`.example.orderExecution.simpleCustomAgg ; (%;(+;`arrivalAskPrice;`endAskPrice);2) ; ` ; 0Nt );
(`strikeToCompletionAskMidPrice ;`simpleCustomAgg ;`.example.orderExecution.simpleCustomAgg ; (%;(+;`arrivalAskPrice;`endAskPrice);2) ; ` ; 0Nt )
);
- Contents of example function:
.example.orderExecution.simpleCustomAgg
.example.orderExecution.simpleCustomAgg:{[OrderAnalyticsRes]
// We can use a util function `.eqea.util.runSimpleAnalytic` to run analytics that are dependent on columns that already exist in the OrderAnalytics table
cfg:select from .eqea.analytics.cfg where analyticType=`simpleCustomAgg;
.eqea.util.simpleAnalytics[OrderAnalyticsRes;cfg]
};
.eqea.util.simpleAnalyticsWithWc
Util function to enable users to run "simple" analytics when a where clause is in the cfg i.e. analytics that make use of columns already present in OrderAnalytics table.
The function .eqea.util.simpleAnalyticsWithWc
runs on a subset of the .eqea.analytics.prevailing.cfg table.
Analytics that make use of .eqea.util.simpleAnalyticsWithWc
should have at a minimum the following values populated in the table passed in to the cfg
argument:
- whereClause
- aggClause
Parameters:
Name | Type | Description |
---|---|---|
OrderAnalyticsRes | table | Table containing all OrderAnalytics data - contains the results of all analytics preceeding the current analytic type. |
cfg | table | Table containing all details required for current batch of analytics, where clause and aggregation clause to apply |
Returns:
Type | Description |
---|---|
table | Table containing all Analytic Results. |
.eqea.util.tickData.getDataAndAggFromCfg
The Util function is used to query tick data and aggregate based on a subset of the .eqea.analytics.cfg table.
Analytics that make use of .eqea.util.tickData.getDataAndAggFromCfg
should have the minimum following values populated in .eqea.analytics.cfg
:
- analytic
- analyticType
- funcName
- aggClause
- marketDataTabName
Parameters:
Name | Type | Description** |
---|---|---|
OrderAnalyticsRes | table | Table containing all OrderAnalytics data - contains the results of all analytics preceeding the current analytic type. |
cfg | table | Subset of .eqea.analytics.cfg - contains all details required for current batch of analytics. |
wcList | list | List of where clauses to be used when querying tick data. NOTE: this list should have the same number of items as there are rows in the Order data. NOTE: if no additional where clause logic is needed then simply use () . |
startTime | symbol | parse tree | Relative start time. Can either be an atomic symbol indictating a timestamp column or a parse tree that conforms to the format: (operator;timeColumn;timestamp), e.g. (-;strikeTime;0D00:01:00), (+; orderCompletedTime;0D00:00:30) |
endTime | symbol | parse tree | Relative end time. Can either be an atomic symbol indictating a timestamp column or a parse tree that conforms to the format: (operator;timeColumn;timestamp), e.g. (-;strikeTime;0D00:01:00), (+; orderCompletedTime;0D00:00:30) |
Returns:
Type | Description |
---|---|
table | Keyed table containing all OrderAnalytic results to this point. |
Example usage:
Writing example custom function .example.orderExecution.tickDataAgg
:
- Populating required values in
.eqea.analytics.cfg
:
.eqea.analytics.cfg:flip `analytic`analyticType`funcName`aggClause`marketDataTabName`joinTimeOffset! flip (
// Examples of adding more complex custom analytics (aggregations on tick data with multiple where clauses)
(`countPriceUnderLimitPrice ;`tickDataAgg ;`.example.orderExecution.tickDataAgg ; (count;`i) ; `Trade ; 0Nt );
(`sumVolumeUnderLimitPrice ;`tickDataAgg ;`.example.orderExecution.tickDataAgg ; (sum;`volume) ; `Trade ; 0Nt )
);
- Contents of example function:
.example.orderExecution.tickDataAgg
.example.orderExecution.tickDataAgg:{[OrderAnalyticsRes]
// We can use `.eqea.util.runTickDataAnalytics` to run analytics on tick data tables
cfg:select from .eqea.analytics.cfg where analyticType=`tickDataAgg;
// Our where clause changes depending on whether the order is a buy or a sell.
// By generating wcList from our input OrderAnalytics table we also ensure the limitPrice value relates to the Order we are analyzing.
wcList:exec whereClause from select whereClause:(((((`BUY`SELL)!(<=;>=)) orderSideCode),\:(`price)),'limitPrice) from OrderAnalyticsRes;
OrderAnalyticsRes:.eqea.util.tickData.getDataAndAggFromCfg[OrderAnalyticsRes;cfg;wcList;`strikeTime;`orderCompletedTime];
OrderAnalyticsRes
};
.eqea.util.tickData.getDataAndAggregate
The Util function can be used to query tick data and aggregate based on input arguments.
This is a low-level function, essentially a wrapper for a functional select, to be used for complex use cases that are incompatible with .eqea.util.tickData.getDataAndAggFromCfg
.
Parameters:
Name | Type | Description |
---|---|---|
tickTabName | symbol | Name of the tick data table to query. |
id | symbol | InstrumentID to query. |
tw | timestamp[2] | Time period of interest. |
wc | list[] | Where clause for query. |
bc | dict / boolean | By clause for query. NOTE: if no by clause needed then use 0b . |
aggDict | dict | Aggregation / column clause for query. |
Returns:
Type | Description |
---|---|
table | Table with the results of our query. |
.eqea.util.asof.ajFromCfg
This function calculates configured analytics that are based on as-of join logic.
This function:
- Builds a time rack for each order based on whats configured in the joinTimeOffset column of the .eqea.analytics.cfg table.
- Performs an as-of join, joining the time rack to the table specified by the marketDataTabName column of the .eqea.analytics.cfg table.
- Pivots the result of the as-of join such that the data is pivoted to a single row per order, under column names as defined by the analytic column of the .eqea.analytics.cfg table.
- Returns the result to be joined onto our input OrderAnalytics data.
Analytics that make use of .eqea.util.asof.ajFromCfg
should have the minimum following values populated in .eqea.analytics.cfg
:
- analytic
- analyticType
- funcName
- aggClause
- marketDataTabName
- joinTimeOffset
Parameters:
Name | Type | Description |
---|---|---|
OrderAnalyticsRes | table | Table containing all OrderAnalytics data - contains the results of all analytics preceding the current analytic type. |
cfg | table | Subset of .eqea.analytics.cfg - contains all details required for current batch of analytics. |
joinOrderTimeCol | symbol | Time column from the Order table. This data what we use as the start point for the time rack table. |
wc | parse tree | Where clause to filter tick data by. |
Returns:
Type | Description |
---|---|
table | Full OrderAnalytics table up to this point. |
Example usage:
Writing example custom function .example.orderExecution.ajExample
:
- Populating required values in
.eqea.analytics.cfg
:
.eqea.analytics.cfg:flip `analytic`analyticType`funcName`aggClause`marketDataTabName`joinTimeOffset! flip (
// Examples which use as of joins
(`myArrivalTradePrice ;`ajExample ;`.example.orderExecution.ajExample ; `price ; `Trade ; 00:00:00 );
(`myArrivalTradePrice_5 ;`ajExample ;`.example.orderExecution.ajExample ; `price ; `Trade ; 00:00:05 );
(`myArrivalTradePrice_10 ;`ajExample ;`.example.orderExecution.ajExample ; `price ; `Trade ; 00:00:10 )
);
- Contents of example function:
.example.orderExecution.ajExample
.example.orderExecution.ajExample:{[OrderAnalyticsRes]
// We can use `.eqea.util.runTickDataAnalytics` to run analytics which retrieve values from tick data using an as of joins
cfg:select from .eqea.analytics.cfg where analyticType=`ajExample;
OrderAnalyticsRes:.eqea.util.asof.ajFromCfg[OrderAnalyticsRes;cfg;`strikeTime;()];
OrderAnalyticsRes
};
.eqea.util.asof.ajTickDataToInput
Efficient as-of join, joining an input data set to tick-data.
It is intended to be used for complex analytics, which are incompatible with .eqea.util.asof.ajFromCfg
.
Parameters:
Name | Type | Description |
---|---|---|
inputData | table | Table containing all timestamps and instrumentIDs of interest. This inputData should have contain columns with names & types matching the primary time and sym columns of the target tick-data. |
tickTabName | symbol | The name of the tick data table we wish to use in our as of join. |
tickDataCols | symbol[] | The column(s) from tabName that we wish to include in our output. Example: If you wanted to get the price and size of a Trade at each time of your input data then tickDataCols would be price size |
wc | parse tree | Where clause to apply to tick data. |
Returns:
Type | Description |
---|---|
table | Result of our as-of join. |
.eqea.util.pivotByCfg
this function pivots a dataset, using the contents of .eqea.analytics.cfg to define output column names.
Ii is intended to be used to pivot data produced by .eqea.util.asof.ajTickDataToInput
.
Parameters:
Name | Type | Description |
---|---|---|
data | table | Table containing Order data which has been asof joined to tick data. |
cfg | table | Subset of .eqea.analytics.cfg - contains all details required for current batch of analytics. |
timeCol | symbol | Time column from the Order table. This is used in conjuction with idCol to perform the pivot. |
idCol | symbol | Sym column from the Order table. This is used in conjuction with timeCol to perform the pivot. |
pivValues | symbol | All columns from tick data containing the data that needs to be pivotted. |
Returns:
Type | Description |
---|---|
table | Pivotted table that can be joined to our OrderAnalytics table. |
.eqea.util.timeRackFromCfg
This function creates a time rack table from inputData based on the joinTimeOffset column from .eqea.analytics.cfg.
The time rack table is generated using joinOrderTimeCol + exec distinct joinTimeOffset from cfg where marketDataTabName=tickTabName
.
It is intended to create a table of timestamps and instrumentIDs as required as an input in .eqea.util.asof.ajTickDataToInput
.
Parameters:
Name | Type | Description |
---|---|---|
inputData | table | Table we want to use for as of join. Usually Order data. |
tickTabName | symbol | Name of the tick data we are joining to. |
cfg | table | Subset of .eqea.analytics.cfg - contains all details required for current batch of analytics. |
joinOrderTimeCol | symbol | Time column from the Order table. This data what we use as the start point for the time rack table. |
Returns:
Type | Description |
---|---|
table | Table with all timestamps and corresponding instrumentIDs as defined by the joinTimeOffset in .eqea.analytics.cfg |