Resources
This section describes the configuration options available under the Resources tab in the kdb Insights Enterprise UI.
When building a database and preparing for deployment, specific resources are allocated to facilitate the process. However, it's important to note that these resources may not be entirely sufficient based on the specific demands of your database.
To ensure a seamless deployment, the Adjust Resources tab, on the database screen gives you options to:
Adjust Resources tab
This tab allows you to define the following resource allocations manually:
Stream
Streams are responsible for moving data between components within kdb Insights Enterprise. They ensure a deterministic order of events from multiple producers in one unified log of the event sequence. The throughput and latency of the system is dependent on the compute resources allocated to a stream compared to how much data is moving through it. The compute resources for a stream allow you to tune your system for your desired throughput.
The following table provides further details for these fields:
| name | description |
|---|---|
| Instances | The number of stream instances. |
| CPU | The request and limit amount of virtual CPU cycles to allocate to this stream. These values are measured in fractions of CPU cores which is either represented as mCPU for millicpu or just simply CPU for whole cores. You can express a half CPU as either 0.5 or as 500mCPU. The limit of CPU available for this stream to consume. During burst operations, this process may consume more than the limit for a short period of time. This is a best effort governance of the amount of compute used. |
| RAM (Memory) | The request and limit amount of RAM to reserve for this process. RAM is measured in bytes using multiples of 1024 (the iB suffix is used to denote 1024 byte multiples). If enough memory is not available on the current infrastructure, Kubernetes will trigger a scale up event on the install. The limit amount of RAM to reserve for this process. If the limit amount RAM is consumed by this stream, it will be evicted and restarted to free up resources. |
| Disk Size | This is the amount of disk allocated to the stream to hold in-flight data. By default, this holds 20GB. This must be tuned to hold enough data so that your system can recover from your maximum allowable downtime. For example, if your data was arriving at 10MB/s and you wanted to allow for a 2 hour recovery, you need at least 70GB of disk space to hold those logs. |
Writedown
This is the amount of compute resources to allocate to the Writedown process.
| name | description |
|---|---|
| CPU | The request and limit amount of virtual CPU cycles to allocate to this instance. This value is measured in fractions of CPU cores which is either represented as mCPU for millicpu or just simply CPU for whole cores. You can express a half CPU as either 0.5 or as 500mCPU. The limit of CPU available for this instance to consume. During burst operations, this process may consume more than the limit for a short period of time. This is a best effort governance of the amount of compute used. |
| RAM (Memory) | The request and limit amount of RAM to reserve for this process. RAM is measured in either bytes using multiples of 1024 (the iB suffix is used to denote 1024 byte multiples). If not enough memory is available on the current infrastructure, Kubernetes triggers a scale up event on the install. The limit amount of RAM to reserve for this process. If the RAM limit is consumed by this instance, it is evicted and restarted to free up resources. |
| Disk Size | The maximum amount of data that can be stored on a given disk drive. |
Query
This is the amount of compute resources to allocate to each of the production and exploratory query instances. Both types of instances use the same compute profile.
Query compute settings allow you to tailor your settings to optimize query performance. These settings are split into Replicas, CPU and RAM (Memory) allocations that can be assigned to the query instances. The request values represent the amount of a resource that is required for compute.
If there are not enough resources available to satisfy the request ask, Kubernetes triggers a scale up event to acquire more resources. Limit values are best effort limits that are applied to query processes. If the limit amount is reached, the query instance is evicted and forced to restart, releasing any resources back to the pool of available resources.
The following table explains these settings.
| name | description |
|---|---|
| Replicas | The number of production and exploratory Replicas. |
| CPU | The request and limit amount of virtual CPU cycles to allocate to this instance. This value is measured in fractions of CPU cores which is either represented as mCPU for millicpu or just simply CPU for whole cores. You can express a half CPU as either 0.5 or as 500mCPU. The limit of CPU available to for this instance to consume. During burst operations, this process may consume more than the limit for a short period of time. This is a best effort governance of the amount of compute used. |
| RAM (Memory) | The request and limit amount of RAM to reserve for this process. RAM is measured in either bytes using multiples of 1024 (the iB suffix is used to denote 1024 byte multiples). If not enough memory is available on the current infrastructure, Kubernetes triggers a scale up event on the install. The limit amount of RAM to reserve for this process. If the RAM limit is consumed by this instance, it is evicted and restarted to free up resources. |
Resource Summary
The Resource Summary, on the right side of the screen, provides a summary of the current resources allocated to the database, under the following headings:
It also gives a summary of total resources in terms of CPUs, RAM and Disk. This resource summary information is also shown in a summary when you click Deploy, as shown below.
Further Reading
- Read how databases are deployed here.