Skip to content

Logging

Logging is a standard tool for system operators to gain observability into their applications. kdb Insights Enterprise utilises industry best-practices when it comes to logging. Log messages are generated in a structured format and emitted to STDOUT where they are consumed by standard logging stacks. Kubernetes collects the log messages into files, where log agents consume them and send them to a downstream log aggregator.

Kubernetes logging.

Configure logging

You can configure the log level of your application at installation time using the values below. kdb Insights Enterprise uses log components as a way to identify the origin of a log message. The routings object allows you to set the log level for different components with DEFAULT applying to anything that's not set explicitly.

global:
  logging:
    routings:
      DEFAULT: INFO
      InfoService: DEBUG

The example above sets all components to INFO with the exception of the InfoService component, which is set to DEBUG.

Logging Stack

kdb Insights Enterprise does not ship with a logging stack as we expect you to choose one based on your use-case and preference. The three main cloud providers (GCP, AWS, and ACS) have managed Kubernetes services which provide the ability to automatically integrate with their logging stacks.

Alternatively you can use a third party agent such as fluent-bit You can configure fluent-bit to forward to any of its supported downstream targets.

Install

Supported targets

KX terraform deploys

If installing your environment with the KX Terraform scripts, the kdb Insights Enterprise Terraform modules are configured by default to deploy and utilize fluent-bit for each of the cloud providers.

Sending logs to other log aggregation systems

The fluent-bit configuration is stored on the following files and can be updated as required:

terraform/k8s_config_aws/logging.tf
terraform/k8s_config_azure/logging.tf
terraform/k8s_config_gcp/logging.tf