How to calculate containers' cpu usage in kubernetes with prometheus as monitoring?

In the Kubernetes environment, using Prometheus to monitor container CPU usage is a highly effective approach. The following are specific steps and practices:

1. Installing Prometheus

First, ensure that Prometheus is installed in your Kubernetes cluster. You can install it in multiple ways, with Helm charts being the most common method.

Installing Prometheus with Helm:

bash
# Add the Prometheus Helm repository
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

# Update the Helm repository
helm repo update

# Install Prometheus
helm install [RELEASE_NAME] prometheus-community/prometheus

This will install Prometheus in your Kubernetes cluster and configure monitoring targets by default.

2. Configuring Prometheus to Monitor Kubernetes

After installation, verify that Prometheus is configured with the correct monitoring targets. Prometheus automatically discovers Kubernetes targets through Service Discovery.

Configuration Example:

yaml
scrape_configs:
  - job_name: 'kubernetes-pods'
    kubernetes_sd_configs:
      - role: pod
    relabel_configs:
      - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
        action: keep
        regex: true
      - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
        action: replace
        target_label: __metrics_path__
        regex: (.+)

This configuration enables Prometheus to monitor all Pods annotated with prometheus.io/scrape: true.

3. Using PromQL to Query CPU Usage

Once Prometheus collects data, you can use PromQL to query specific CPU usage metrics.

Querying Pod CPU Usage:

plaintext
rate(container_cpu_usage_seconds_total{container!="POD",namespace="default"}[5m])

This PromQL statement calculates the CPU usage rate over the past 5 minutes for each container in the default namespace.

4. Using Grafana to Display Data

For enhanced visualization and monitoring, use Grafana to display Prometheus-collected data. Grafana connects to the Prometheus data source and builds graphical dashboards.

Connecting Grafana to Prometheus:

In Grafana, add Prometheus as a data source and input the Prometheus service URL.

Creating a Dashboard to Display CPU Usage:

In Grafana, create a new dashboard, add a chart, and configure the PromQL query statement to visualize CPU usage.

Summary

By following these steps, you can effectively monitor and query container CPU usage in Kubernetes using Prometheus. Combined with Grafana, this approach provides an intuitive monitoring experience. This method helps operations teams better understand resource utilization and optimize resource allocation in a timely manner.