Skip to content

Database

Create a database using the UI. A database is used to store the data you will need to ingest. Important is knowing the structure of incoming data with respect to column header names and desired kdb+ datatypes. Free tier comes with an insights-demo database, it just needs to be deployed; insights-demo is listed under the Data menu in the entity tree. New users should first create a database using the wizard.

Screenshot

Set up

  1. Select between Time-series or Custom database.

  2. Define database name; Submit.

  3. Custom databases have a standard or advanced configuration for a Mount; in advanced mode, configure additional details for Mount, Data Access and Storage. Default settings are sufficient for creating a new database.

  4. Submit

Database wizard

Time-series

A quick start option for creating a database with a predefined time-series configuration.

Database Name
Name of database.
Advanced
Toggle for additional database configuration options.

Screenshot

Custom

A standard database with custom Mounts.

Advanced
Toggle for additional database configuration options.
Database Name
Name of database.

Screenshot

Mount

item description
Name Name of mount, e.g. rdb.
Type Select from stream, local, object or purge.
Partition Partition scheme for the mount, e.g. none, ordinal, date, month, year.
Dependencies List of mounts required to be mounted as a dependency of this mount.
BaseURI Uniform Resource Identifier (URI) representing where data can be mounted by other services; set to none.
Frequency A timespan in q notation. How often should this tier roll data over into the next tier, e.g. 00:00:00 or 0D00:00:00.
Snap A time in q notation. At what whole multiples of time should rollovers be scheduled, e.g. 00:00:00 or 0D00:00:00.
Retain A timespan consisting of a number followed by a unit: (Years, Months, Weeks, Days, Hours, Minutes or Seconds), e.g. 2 Years. Rollover occurs for data which has been stored for this length of time.

Advanced

There are no required configurations for a database in the Advanced menu.

Define Mounts

Screenshot

item description
Name Name of mount, e.g. rdb.
Type Select from stream, local, object or purge.
Partition Partition scheme for the mount, e.g. none, ordinal, date, month, year.
Dependencies List of mounts required to be mounted as a dependency of this mount.
BaseURI Uniform Resource Identifier (URI) representing where data can be mounted by other services; set to none.
Description A description of the purpose of the mount.
Volume Requested PV size.

Add Instances

Screenshot

item description
Name Name of data access mount, e.g. rdb.
Custom File Path to custom file loaded by DA instance'; e.g. /full/path/file.q or mount name, e.g. $volumemount/file.q.
RT Logs volume Set requested PV size, e.g. 20Gi.
Size Size is the size of the Data Access Deployment; default to 3
Source Sequencer Bus to subscribe too.

Tiers

Add and configure Tier properties

Screenshot

item description
Mount The name of the corresponding mount at which data in this tier can be accessed, e.g. rdb.
Name Reference to a particular tier.
Description (Toggle dropdown). Description of tier purpose.
Store (Toggle dropdown). A URI describing where this tier will physically store data.
Compression

Optional compression policy for data.

item description
Algorithm Select algorithm to apply from dropdown list; defaults to none.
Block Define block size for compression; numeric value.
Level Set compression level; numeric value.
Retain

Policy for how much data should be stored in this tier before it rolled over into the next tier.

item description
Rows Rollover occurs for data beyond the number of rows set by this variable.
Size Threshold of tier size before rollover occurs. Number in byte size followed by unit, e.g. 2 TB.
Size Pct Threshold of tier size before rollover occurs, expressed as a percentage of total storage for the corresponding mount; value between 1 and 100.
Time Time threshold for rollover of data stored for this length of time. A timespan consisting of a number followed by a unit, e.g. 2 Years.
Schedule

Policy for when rollovers should be considered.

item description
Frequency A timespan in q notation. How often should this tier roll data over into the next tier.
Snap A time in q notation. At what whole multiples of time should rollovers be scheduled.

Storage

item description
Enforce Schema Enable to enforce table schema when persisting - used for debugging, may incur performance penalty.
EOD Peach Level Level at which EOD peaches to parallelize HDB table processing; select between part or table.
Rt Logs Volume Size RT logs volume requested PV size, e.g. 20 Gi.
Size Size of the Storage Manager deployment.
Sort Limit GB Memory limit when sorting splayed tables or partitions on disk in GB.
Source Either a URI pointed at a static data source of the name of an entry in bus from which to obtain streaming data.