grafana prometheus 可视化监控中心

Spring Boot 监控平台 Prometheus + Grafana 入门

基于Prometheus构建MySQL可视化监控平台

基于Prometheus构建黑盒监控体系

PrometheusAlert 开源的运维告警中心消息转发系统

grafana 安装：

docker stop grafana
docker rm grafana
docker run -d \
      -p 3000:3000 \
      --name=grafana \
      -e "GF_SECURITY_ADMIN_PASSWORD=admin" \
      -e "GF_INSTALL_PLUGINS=grafana-clock-panel,grafana-simple-json-datasource,raintank-worldping-app,grafana-piechart-panel" \
      -v $PWD/data:/var/lib/grafana \
      grafana/grafana

prometheus 安装：

docker pull prom/prometheus
docker run -d --name prometheus -p 9090:9090 prom/prometheus

mkdir conf
mkdir data
chmod 777 -R data

docker cp prometheus:/etc/prometheus/prometheus.yml conf/

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
    - targets: ['localhost:9090']
      labels:
        instance: prometheus

docker stop prometheus
docker rm prometheus
docker run \
        -p 9090:9090 \
        -v $PWD/conf/prometheus.yml:/etc/prometheus/prometheus.yml \
        -v $PWD/data:/prometheus \
        --name prometheus \
        -d prom/prometheus \

node-exporter 安装 （服务器监控）

docker run -d \
        --name node-exporter \
        -p 9100:9100 \
        -v "/proc:/host/proc" \
        -v "/sys:/host/sys" \
        -v "/:/rootfs" \
        --net="host" \
        prom/node-exporter

# 修改prometheus的配置
- job_name: 'node'
    static_configs:
    - targets: ['192.168.28.130:9100']
      labels:
        env: test
        name: node1
        instance: 192.168.28.130

cadvisor 安装 （docker监控）

docker run \
	-v /var/run:/var/run:rw  \
	-v /sys:/sys:ro  \ 
	-v /var/lib/docker:/var/lib/docker:ro \ 
	-p 8080:8080  \
	-d  --name cadvisor google/cadvisor

Alertmanager 安装 （报警）

docker run -d -p 9093:9093 -v $PWD/data/config.yml:/etc/alertmanager/config.yml --name alertmanager prom/alertmanager

#config.yml内容
global:
  smtp_smarthost: 'smtp.163.com:25'　　#163服务器
  smtp_from: '[email protected]'　　　　　　　　#发邮件的邮箱
  smtp_auth_username: '[email protected]'　　#发邮件的邮箱用户名，也就是你的邮箱
  smtp_auth_password: 'XXX'　　　　　　　　#发邮件的邮箱密码

route:
  group_by: ['alertname']

  repeat_interval: 1h

  receiver: live-monitoring

receivers:
- name: 'live-monitoring'
  email_configs:
  - to: '[email protected]'　　　　　　　　#收邮件的邮箱
 

# 修改prometheus配置文件prometheus.yml，开启报警功能，添加报警规则配置文件
# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets: ["localhost:9093"]
      # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  - "node_down.yml"
  - "memory_over.yml"
 
# node_down.yml内容
groups:
- name: example
  rules:
  - alert: InstanceDown
    expr: up == 0
    for: 1m
    labels:
      user: caizh
    annotations:
      summary: "Instance {{ $labels.instance }} down"
      description: "{{ $labels.instance }} of job {{ $labels.job }} has been down for more than 1 minutes."

# memory_over.yml 内容
groups:
- name: example
  rules:
  - alert: NodeMemoryUsage
    expr: (node_memory_MemTotal_bytes - (node_memory_MemFree_bytes+node_memory_Buffers_bytes+node_memory_Cached_bytes )) / node_memory_MemTotal_bytes * 100 > 80
    for: 1m
    labels:
      user: caizh
    annotations:
      summary: "{{$labels.instance}}: High Memory usage detected"
      description: "{{$labels.instance}}: Memory usage is above 80% (current value is:{{ $value }})"

# 重启 prometheus
docker run -d -p 9090:9090 --name=prometheus \
-v $PWD/data/prometheus.yml:/etc/prometheus/prometheus.yml \
-v $PWD/data/node_down.yml:/etc/prometheus/node_down.yml \
-v $PWD/data/memory_over.yml:/etc/prometheus/memory_over.yml \
prom/prometheus 

参考文档：
docker 安装 grafana