Skip to content

Pod annotation-based monitoring

Pod annotation-based monitoring is a configuration mechanism used within Kubernetes environments to automate the discovery and scraping of custom application metrics.^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md]

This method leverages the standard Kubernetes metadata system to allow applications to explicitly signal monitoring tools—specifically Prometheus—about their availability and configuration details.^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md]

Mechanism

In this pattern, specific annotations are applied to the metadata of a Pod definition. The monitoring server, configured with a service discovery role for pods (e.g., kubernetes_sd_configs: role: pod), inspects these annotations to determine if and how to scrape metrics from the container.^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md]

Common Annotations

The following annotations are typically used to control the scraping behavior:

  • Scraping Enablement: The annotation prometheus_io_scrape: "true" signals to the monitoring system that the target should be scraped.^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md]
  • Metrics Path: The prometheus_io_path annotation specifies the HTTP path where metrics are exposed (e.g., /metrics or /).^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md]
  • Port Configuration: The prometheus_io_port annotation defines the port number on which the metrics endpoint is available.^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md]

Prometheus Configuration

On the Prometheus server side, this workflow is enabled through relabel_configs within the prometheus.yml configuration file.^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md]

These configurations perform the following actions:

  1. Filtering: They use the keep action to process only pods that have the prometheus_io_scrape annotation set to true.^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md]
  2. Path Replacement: They extract the value from prometheus_io_path and set the __metrics_path__ label accordingly.^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md]
  3. Address Replacement: They construct the final scrape address by combining the Pod's IP address with the port specified in the prometheus_io_port annotation.^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md]

Sources