Skip to content

Kubernetes log collection

Kubernetes log collection is the process of aggregating and analyzing log data generated across a Kubernetes cluster.^[400-devops-07-monitoring-and-observability-fluent-bit.md]

Common Architecture

A standard approach to collecting logs in Kubernetes involves the EFK stack, which consists of Elasticsearch, Fluent Bit (or Fluentd), and Kibana.^[400-devops-07-monitoring-and-observability-fluent-bit.md]

Components

  • Elasticsearch: A search and analytics engine used to store the log data.
  • Fluent Bit / Fluentd: Log forwarding agents deployed on the cluster to collect, parse, and ship logs to the backend storage^[400-devops-07-monitoring-and-observability-fluent-bit.md].
  • Kibana: A visualization tool that allows users to explore the data stored in Elasticsearch^[400-devops-07-monitoring-and-observability-fluent-bit.md].

Tools

  • Fluent Bit: A fast and lightweight log processor and forwarder, commonly used in Kubernetes environments^[400-devops-07-monitoring-and-observability-fluent-bit.md].
  • Fluentd: A unified logging layer, often used interchangeably or in conjunction with Fluent Bit^[400-devops-07-monitoring-and-observability-fluent-bit.md].

Sources

^[400-devops-07-monitoring-and-observability-fluent-bit.md]