Skip to content

Alertmanager configuration and routing

Alertmanager is a component used in conjunction with monitoring systems like Prometheus to handle alert routing, deduplication, and grouping. It manages notifications by receiving alerts from the server and dispatching them to configured receivers[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md][400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md].

Core Concepts

Functionality

The Alertmanager acts as an intermediary between the monitoring server (e.g., Prometheus) and the end receivers. It performs several key logical operations on incoming alerts^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md]:

  • Deduplication: Removes duplicate alerts to prevent spamming.
  • Grouping: Aggregates related alerts into a single notification.
  • Routing: Directs alerts to specific receivers based on configuration rules.

Common receivers include email, PagerDuty, or other integrations^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md].

Integration

In a standard setup, the monitoring server is configured to send fired alerts to the Alertmanager. For example, in Prometheus configuration, this is done under the alerting section by specifying the Alertmanager's address as a target^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md].

Configuration Structure

Alertmanager behavior is defined via a configuration file (often config.yml or alertmanager.yml). This configuration is typically mounted into the container, for example, via a Kubernetes ConfigMap^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md].

Global Settings

The global section defines default parameters that apply to all alerts and receivers unless overridden^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md].

  • Resolve Timeout: Sets the time duration after which an alert is declared as resolved if no updates are received^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md].
  • SMTP Settings: Configures email server details for email notifications, including:
    • smtp_smarthost: The mail server address and port^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md].
    • smtp_from: The sender email address^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md].
    • smtp_auth_username & smtp_auth_password: Credentials for authentication^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md].
    • smtp_require_tls: Specifies whether TLS is required^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md].

Route Configuration

The route block defines the routing tree and the strategy for how alerts are grouped and sent^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md].

  • Receiver: The default receiver (e.g., 'default') to which alerts are sent if no other route matches^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md].
  • Group By: A list of labels used to aggregate alerts. For example, ['alertname', 'cluster'] creates a group for alerts sharing these specific label values^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md].
  • Group Wait: The time to wait before sending the first notification for a new group of alerts. This allows time for more related alerts to arrive, batching them into a single message^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md].
  • Group Interval: The time to wait before sending a notification about new alerts that are added to an existing group for which an initial notification has already been sent^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md].
  • Repeat Interval: The time to wait before re-sending a notification for an alert that is still firing and has already been notified^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md].

Receivers

The receivers section defines the named destinations for alerts^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md].

  • Name: A unique identifier (e.g., 'default') referenced by the route block^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md].
  • Email Configs: A list of email configurations. Key parameters include:
    • to: The recipient email address^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md].
    • send_resolved: If set to true, the receiver will also be notified when the alert status returns to "resolved"^[400-devops__06-Kubernetes__k8s-paas__07.Promtheus监控k8s企业级应用.md].

Sources