Intraday database (IDB)

For systems in which memory is a constraint for the daily data volume, the IDB process can be used. By adding an IDB to the pipeline configuration, data will flush from the RDB and be available from disk via the IDB on each PDB write. The PDB write is determined by write-row-limit and write-freq.

There are no special configuration flags needed for the IDB in the pipeline YAML as it is effectively an HDB looking at a small subset of data.

Warning

When creating a pipeline with only Intra-Persist type None tables, don't include an IDB in your pipeline as there will be no intraday write-downs.