Skip to content

Schema

Schemas can be created independently or as part of the data assembly wizard builder. A schema transforms data to the correct datatype of the destination database.

Additional information on Table configuration can be found here.

Screenshot

Set up

  1. Name the schema

  2. Starting with the default table1 or with a new table (+), add data columns corresponding to the data ingested; define the name and kdb type for each column of data; columm description is optional. Sample kdb types for ingested data include:

    item column type
    time column Date, Datetime, Time, Timestamp
    category (independent data) Symbol, String
    numeric (dependent data) Float, Integer
  3. Define Table properties. Required is a definition for the Timestamp Column, which will be the time column of the data. The Partition column can also be defined. Define the data sort for interval, real-time and historical data; typically a sort is done using the aforementioned time column.

  4. Submit

Create

item description
Schema Name Name the schema to be created
Advanced Toggle additional properties for the schema
Submit Submit a completed schema

Properties

Every schema has a set of properties. Every schema requires a Timestamp Column definition and typically a Partition Column too.

Screenshot

item description
Primary Keys A list of primary key column names.
Description Description of the table purpose.
Type (Advanced) Select between splayed or partitioned.
Partition Column Select from dropdown options; defaults to none.
Shards (Advanced) Set number of shards.
Partitions (Advanced) Set number of partitions.
Block Size (Advanced) Set block size.
Timestamp Column (required) Select from dropdown options; defaults to none.
Real-time Sort Real-time (rdb) data sort column
Interval Sort Interval (idb) data sort column
Historical Sort Historical (hdb) data sort column

Columns

Click [+] to add a column, delete from the Screenshot menu to remove one. Columns can be repositioned on click-and-drag. Select the expander for additional properties: Primary Key, On-Disk Attributes and Foreign.

Screenshot

item description
Name Assign a name to the column in the table.
Type Select the data type for the column; boolean, guid, byte, short, int, long, real, float, char, symbol, timestamp, month, date, datetime, timespan, minute, second or time.
Description Optional description of the column purpose.
Primary Key Check if a Primary Key column.
On-Disk Attribute (Temporal Partitioning) Column attribute when stored on disk. Select from None, Sorted, Grouped, Parted and Unique.
In-Memory Attribute Column attribute when stored in memory. Select from None, Sorted, Grouped, Parted and Unique.
On-Disk Attribute (Ordinal Partitioning) Column attribute when stored on disk with an ordinal partition scheme. Select from None, Sorted, Grouped, Parted and Unique.
Foreign A foreign key into another table in this Assembly of the form table.column.

Schemas Require a Timestamp Partition Column

Schemas require have a time data column. In addition, the table is typically partitioned and sorted (interval, historic and/or real-time) by this time column. This can be configured as part of the set up wizard, or with the essential and advanced properties of a schema.

Storage Manager Tables Configuration