Skip to content

Prometheus-based K8S Monitoring Stack

The Prometheus-based K8S Monitoring Stack is a comprehensive monitoring solution designed to track the health and performance of both the Kubernetes cluster infrastructure and the enterprise applications running within it.^[400-devops__06-Kubernetes__k8s-paas__README.md] This stack typically integrates Prometheus as the core Metrics gathering system alongside visualization and alerting tools.

Architecture and Components

To achieve full observability, the stack relies on several key components deployed within the K8S environment^[400-devops__06-Kubernetes__k8s-paas__README.md]:

  • Core System: Prometheus-server acts as the central processing unit, responsible for scraping and storing metrics^[400-devops__06-Kubernetes__k8s-paas__README.md].
  • Visualization: Grafana is used to install, deploy, and configure dashboards for real-time data analysis^[400-devops__06-Kubernetes__k8s-paas__README.md].
  • Alerting: Alertmanager is installed and deployed to handle the routing of alerts based on specific rules^[400-devops__06-Kubernetes__k8s-paas__README.md].
  • Logging: The stack often integrates with an ELK architecture (Elasticsearch, Logstash, Kibana) to handle log aggregation and analysis alongside metrics^[400-devops__06-Kubernetes__k8s-paas__README.md].

Data Collection Agents

The monitoring stack employs various exporters to gather different types of telemetry data^[400-devops__06-Kubernetes__k8s-paas__README.md]:

  • kube-state-metrics: Focuses on monitoring the state of Kubernetes objects (e.g., deployment status, Pod health)^[400-devops__06-Kubernetes__k8s-paas__README.md].
  • node-exporter: Collects hardware and OS-level Metrics from the cluster nodes^[400-devops__06-Kubernetes__k8s-paas__README.md].
  • cAdvisor: Provides container resource usage statistics, such as CPU and memory utilization^[400-devops__06-Kubernetes__k8s-paas__README.md].
  • blackbox-exporter: Used for probing endpoints over HTTP, HTTPS, or ICMP to check external reachability^[400-devops__06-Kubernetes__k8s-paas__README.md].

Application Monitoring

Beyond infrastructure Metrics, the stack is configured to monitor business containers and services^[400-devops__06-Kubernetes__k8s-paas__README.md]. This involves configuring Prometheus to scrape Metrics directly from application workloads. For complex logging scenarios involving microservices, such as a Dubbo consumer, the stack may utilize Filebeat to collect logs and stream them to the ELK architecture^[400-devops__06-Kubernetes__k8s-paas__README.md].

Sources

  • 400-devops__06-Kubernetes__k8s-paas__README.md