5.1.0

Release Date: 2021.03.03

Refinery Core

What's new

  • Intraday Database (IDB)
    • Allows intraday data to be accessed from on-disk rather than having to have all intraday data in memory
  • PDB: Now will take into account par.txt in HDB root to support segmented HDBs

Enhancements

  • Added support for Platform 4.6
  • Shared library support for CentOS 8 and Ubuntu 20.04
  • TP: Can receive updates via 'upd' as well as '.u.upd'

Non-upwardly compatible changes

System schemas can no longer be defined in routing -> custom -> data-types

Schemas in routing -> custom -> data-types

In 5.0.0 schemas defined within the system could also be added to the custom routing configuration for the pipeline:

routing:
  custom:
    data-types:
      - taq           <- Not a schema (OK)
      - taqWide       <- Not a schema (OK)
      - CDR_SpeedTest <- A schema (BAD)

This would bypass the data availability checks within the Gateway and cause it to route to a HDB before any data was present.

The only recommended use for the custom routing configuration is for data types that do not directly map to a schema or a schema that is not defined via YAML.

The Process Manager has been patched such that any YAML-defined schema in the custom routing configuration will cause it to throw a InvalidRoutingConfiguration exception on boot.