Logging is a standard tool for system operators to gain observability into their applications. kdb Insights Enterprise utilises industry best-practices when it comes to logging. Log messages are generated in a structured format and emitted to STDOUT where they are consumed by standard logging stacks. Kubernetes collects the log messages into files, where log agents consume them and send them to a downstream log aggregator.
You can configure the log level of your application at installation time using the values below.
kdb Insights Enterprise uses log components as a way to identify the origin of a log message.
routings object allows you to set the log level for different components
DEFAULT applying to anything that's not set explicitly.
global: logging: routings: DEFAULT: INFO InfoService: DEBUG
The example above sets all components to
INFO with the exception of the
InfoService component, which is set to
kdb Insights Enterprise does not ship with a logging stack as we expect you to choose one based on your use-case and preference. The three main cloud providers (GCP, AWS, and ACS) have managed Kubernetes services which provide the ability to automatically integrate with their logging stacks.
Alternatively you can use a third party agent such as fluent-bit
You can configure
fluent-bit to forward to any of its supported downstream targets.
KX terraform deploys
If installing your environment with the KX Terraform scripts, the kdb Insights Enterprise Terraform modules are configured by default to deploy and utilize fluent-bit for each of the cloud providers.
Sending logs to other log aggregation systems
The fluent-bit configuration is stored on the following files and can be updated as required:
terraform/k8s_config_aws/logging.tf terraform/k8s_config_azure/logging.tf terraform/k8s_config_gcp/logging.tf