Skip to content

Filebeat Sidecar Pattern

The Filebeat Sidecar Pattern is a deployment strategy in containerized environments where a log shipping agent runs as a companion container within the same Pod as the main application container.^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md]

Architecture

In this pattern, the application container and the Filebeat container share a common storage volume.^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md] This tight coupling ensures that Filebeat has immediate access to the log files generated by the application, facilitating real-time log collection.

Implementation

The configuration typically involves defining a Deployment resource that specifies two containers within the same spec:^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md]

  • Application Container: Generates the application logs.
  • Filebeat Container: Consumes the logs from the shared volume and forwards them.

A shared emptyDir volume is mounted into both containers. The application writes logs to this volume, and Filebeat reads from the same path.^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md]

Data Flow

Logs collected by Filebeat are published to a centralized log processing pipeline. In a typical enterprise setup, Filebeat pushes data to Kafka, which acts as a buffer.^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md] Downstream consumers like Logstash then process the data from Kafka before indexing it in Elasticsearch.^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md]

  • [[ELK Stack|ELK]]
  • [[Kafka]]
  • [[Logstash]]

Sources