System overview (UI)
This is a high-level account for importing data - writing to a database - then querying and visualizing in kdb Insights Enterprise. Get familiar with terms and concepts used in the UI with a helpful glossary, then take the interactive guided tour to put your knowledge into practice.
1: Create a database
Data is stored in kdb Insights Enterprise using kdb+, column-based, relational time series database technology. Simply pick a database size and you're ready to write data to it.
2: Create a schema
A schema contains table definitions to ensure imported data is compatible with kdb+ data types. You must create a schema for the data that you want to import. This is done manually, or using a JSON file, as part of the database.
3: Create a pipeline
A pipeline is the process that ingests data from a source into kdb+. The process is defined in a pipeline template. The simplest template comprises three nodes:
- A reader - reads data from a source
- A transformer - applies a schema to import data
- A writer - writes data to the kdb Insights Enterprise database.
Other nodes are available.
You must create one or more pipeline templates to import data.
More about creating a pipeline
4: Import data
To import data, you must:
-
Deploy the database
-
Deploy at least one pipeline
Data ingestion begins when you deploy a pipeline. You can deploy several pipelines at once, or, if resources are limited, one at a time, tearing down each pipeline after the import.
More about importing data via the import wizard
5: Query data
You can query imported data using q, SQL or Python. View results in the console window, a formatted table, or a simple chart.
6: Visualize data
Imported data tables are available in a visualization tool. You can incorporate data into charts, maps and more, and share these views with colleagues.