Skip to content

Sidecar logging patterns

In Kubernetes environments, cluster-level-logging architectures are designed to ensure log collection is decoupled from the lifecycle of Pods, Nodes, or containers.^[400-devops__06-Kubernetes__k8s-paas__原理及源码解析__Kubernetes相关生态.md] This ensures that logs remain accessible even if a container crashes or a Node fails.^[400-devops__06-Kubernetes__k8s-paas__原理及源码解析__Kubernetes相关生态.md]

While a common baseline pattern involves deploying a central logging agent on every Node (e.g., as a DaemonSet) to read from stdout and stderr, this approach has limitations.^[400-devops__06-Kubernetes__k8s-paas__原理及源码解析__Kubernetes相关生态.md] Specifically, high-volume logs can exhaust system quotas, or applications may be configured to write exclusively to log files rather than standard streams.^[400-devops__06-Kubernetes__k8s-paas__原理及源码解析__Kubernetes相关生态.md]

Sidecar Patterns

When applications write logs to files instead of standard output, or when direct stdout logging is insufficient, sidecar containers are introduced into the Pod to handle the log stream.

1. Stream Redirection Sidecar

This pattern involves deploying a sidecar container alongside the main application to retrieve log files and redirect them back to the sidecar's own stdout and stderr streams.^[400-devops__06-Kubernetes__k8s-paas__原理及源码解析__Kubernetes相关生态.md]

By streaming to stdout and stderr, this pattern allows the pre-existing Node-level logging agent to capture and forward the logs without requiring changes to the Node's agent configuration.^[400-devops__06-Kubernetes__k8s-paas__原理及源码解析__Kubernetes相关生态.md]

Trade-offs

  • Compatibility: It enables compatibility for applications that can only log to files, integrating them into the standard stdout-based logging pipeline^[400-devops__06-Kubernetes__k8s-paas__原理及源码解析__Kubernetes相关生态.md].
  • Storage Overhead: A significant disadvantage is disk waste, as the log data is stored twice: once in the original application log file and again in the log file corresponding to the sidecar's stdout/stderr JSON output^[400-devops__06-Kubernetes__k8s-paas__原理及源码解析__Kubernetes相关生态.md].
  • Recommendation: Due to the storage inefficiency, this pattern is generally discouraged unless the application container cannot be modified^[400-devops__06-Kubernetes__k8s-paas__原理及源码解析__Kubernetes相关生态.md].

2. Streaming Agent Sidecar

In this pattern, the sidecar container runs a dedicated logging agent (such as fluentd) that reads the application's log files and streams them directly to a remote backend (e.g., ElasticSearch)^[400-devops__06-Kubernetes__k8s-paas__原理及源码解析__Kubernetes相关生态.md]. This bypasses the Node-level logging agent and standard output entirely.

Trade-offs

  • Resource Consumption: The sidecar agent may consume significant compute resources, potentially impacting the performance of the main application container^[400-devops__06-Kubernetes__k8s-paas__原理及源码解析__Kubernetes相关生态.md].
  • Visibility: Because logs do not pass through stdout or stderr, they are invisible to standard Kubernetes tooling like kubectl logs^[400-devops__06-Kubernetes__k8s-paas__原理及源码解析__Kubernetes相关生态.md].

Operational Considerations

Regardless of the specific logging pattern implemented, managing log retention on the host is critical^[400-devops__06-Kubernetes__k8s-paas__原理及源码解析__Kubernetes相关生态.md]. Log files must be cleaned periodically or stored on remote volumes; otherwise, filling the main disk partition can lead to system crashes^[400-devops__06-Kubernetes__k8s-paas__原理及源码解析__Kubernetes相关生态.md].

Sources

^[400-devops__06-Kubernetes__k8s-paas__原理及源码解析__Kubernetes相关生态.md]