Logging
Logging is a standard tool for system operators to gain observability into their applications. kdb Insights Enterprise utilises industry best-practices when it comes to logging. Log messages are generated in a structured format and emitted to STDOUT where they are consumed by standard logging stacks. Kubernetes collects the log messages into files, where log agents consume them and send them to a downstream log aggregator.
Configure logging
You can configure the log level of your application at installation time using the values below.
kdb Insights Enterprise uses log components as a way to identify the origin of a log message.
The routings object allows you to set the log level for different components
with DEFAULT applying to anything that's not set explicitly.
global:
logging:
routings:
DEFAULT: INFO
InfoService: DEBUG
The example above sets all components to INFO with the exception of the InfoService component, which is set to DEBUG.
Logging Stack
kdb Insights Enterprise does not ship with a logging stack as we expect you to choose one based on your use-case and preference. The three main cloud providers (GCP, AWS, and ACS) have managed Kubernetes services which provide the ability to automatically integrate with their logging stacks.
Alternatively you can use a third party agent such as fluent-bit
You can configure fluent-bit to forward to any of its supported downstream targets.
KX terraform deploys
If installing your environment with the KX Terraform scripts, the kdb Insights Enterprise Terraform modules are configured by default to deploy and utilize fluent-bit for each of the cloud providers.
Sending logs to other log aggregation systems
The fluent-bit configuration is stored on the following files and can be updated as required:
terraform/k8s_config_aws/logging.tf
terraform/k8s_config_azure/logging.tf
terraform/k8s_config_gcp/logging.tf