Prometheus配置alertmanager告警钉钉机器人通知

前3步和这里的前3步一样：Prometheus配置alertmanager告警邮件通知

4. docker安装prometheus-webhook-dingtalk

建议先不映射路径，进去把/etc/prometheus-webhook-dingtalk整个文件夹cp出来，再改配置会方便点

docker run --name prometheus-webhook-dingtalk \
    -p 0.0.0.0:8060:8060 \
    -v /etc/prometheus-webhook-dingtalk:/etc/prometheus-webhook-dingtalk \
    --restart always \
    -itd timonwong/prometheus-webhook-dingtalk:latest \

5. 编写alertmanager配置

位置：/etc/alertmanager/alertmanager.yml

global:  
  resolve_timeout: 5m  
route:  
  receiver: webhook  
  group_wait: 10s  
  group_interval: 15s  
  repeat_interval: 30s  
  group_by: [alertname]  
  routes:  
  - receiver: webhook  
    group_wait: 10s  
receivers:  
- name: webhook  
  webhook_configs:  
  - url: http://192.168.7.254:8060/dingtalk/webhook1/send  
    send_resolved: true

6. 编写prometheus-webhook-dingtalk配置

位置：/etc/prometheus-webhook-dingtalk/config.yml
只需要没注释的那几行

## Request timeout
# timeout: 5s

## Uncomment following line in order to write template from scratch (be careful!)
#no_builtin_template: true

## Customizable templates path
templates:
  - /etc/prometheus-webhook-dingtalk/templates/templates.tmpl

## You can also override default template using `default_message`
## The following example to use the 'legacy' template from v0.3.0
#default_message:
#  title: '{{ template "legacy.title" . }}'
#  text: '{{ template "legacy.content" . }}'

## Targets, previously was known as "profiles"
targets:
  webhook1:
    url: https://oapi.dingtalk.com/robot/send?access_token=66666666666666666666666666666666666666666666
    # secret for signature
    secret: SEC66666666666666666666666666666666666666666666
    message:
      text: '{{ template "ding.link.content" . }}'
#  webhook2:
#    url: https://oapi.dingtalk.com/robot/send?access_token=xxxxxxxxxxxx
#  webhook_legacy:
#    url: https://oapi.dingtalk.com/robot/send?access_token=xxxxxxxxxxxx
#    # Customize template content
#    message:
#      # Use legacy template
#      title: '{{ template "legacy.title" . }}'
#      text: '{{ template "legacy.content" . }}'
#  webhook_mention_all:
#    url: https://oapi.dingtalk.com/robot/send?access_token=xxxxxxxxxxxx
#    mention:
#      all: true
#  webhook_mention_users:
#    url: https://oapi.dingtalk.com/robot/send?access_token=xxxxxxxxxxxx
#    mention:
#      mobiles: ['156xxxx8827', '189xxxx8325']

7. 编写模板配置

位置：/etc/prometheus-webhook-dingtalk/templates/templates.tmpl
用到的是ding.link.content模板

{{ define "__subject" }}[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}] {{ .GroupLabels.SortedPairs.Values | join " " }} {{ if gt (len .CommonLabels) (len .GroupLabels) }}({{ with .CommonLabels.Remove .GroupLabels.Names }}{{ .Values | join " " }}{{ end }}){{ end }}{{ end }}
{{ define "__alertmanagerURL" }}{{ .ExternalURL }}/#/alerts?receiver={{ .Receiver }}{{ end }}

{{ define "__text_alert_list" }}{{ range . }}
**Labels**
{{ range .Labels.SortedPairs }}> - {{ .Name }}: {{ .Value | markdown | html }}
{{ end }}
**Annotations**
{{ range .Annotations.SortedPairs }}> - {{ .Name }}: {{ .Value | markdown | html }}
{{ end }}
**Source:** [{{ .GeneratorURL }}]({{ .GeneratorURL }})
{{ end }}{{ end }}

{{ define "default.__text_alert_list" }}{{ range . }}
#### \[{{ .Labels.severity | upper }}\] {{ .Annotations.summary }}

**Description:** {{ .Annotations.description }}

**Graph:** []({{ .GeneratorURL }})

**Details:**
{{ range .Labels.SortedPairs }}{{ if and (ne (.Name) "severity") (ne (.Name) "summary") }}> - {{ .Name }}: {{ .Value | markdown | html }}
{{ end }}{{ end }}
{{ end }}{{ end }}

{{/* Default */}}
{{ define "default.title" }}{{ template "__subject" . }}{{ end }}
{{ define "default.content" }}#### \[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}\] **[{{ index .GroupLabels "alertname" }}]({{ template "__alertmanagerURL" . }})**
{{ if gt (len .Alerts.Firing) 0 -}}
**Alerts Firing**
{{ template "default.__text_alert_list" .Alerts.Firing }}
{{- end }}
{{ if gt (len .Alerts.Resolved) 0 -}}
**Alerts Resolved**
{{ template "default.__text_alert_list" .Alerts.Resolved }}
{{- end }}
{{- end }}

{{/* Legacy */}}
{{ define "legacy.title" }}{{ template "__subject" . }}{{ end }}
{{ define "legacy.content" }}#### \[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}\] **[{{ index .GroupLabels "alertname" }}]({{ template "__alertmanagerURL" . }})**
{{ template "__text_alert_list" .Alerts.Firing }}
{{- end }}



{{ define "__ding_text_alert_list" }}{{ range . }}
#### **{{ .Labels.name }}**
> - **告警:** {{ .Labels.alertname }}
> - **目标:** {{ .Labels.instance }}
> - **配置:** {{ .Labels.job }}
{{ end }}{{ end }}


{{/* Following names for compatibility */}}
{{ define "ding.link.content" }}
  {{ if gt (len .Alerts.Firing) 0 -}}
    ## **告警: {{ .Alerts.Firing | len }} 项**
    
    {{ template "__ding_text_alert_list" .Alerts.Firing }}
  {{- end }}

  {{ if gt (len .Alerts.Resolved) 0 -}}
    ## **已恢复: {{ .Alerts.Resolved | len }} 项**{{ if eq (len .Alerts.Firing) 0 -}}（已全部恢复）{{- end }}
    
    {{ template "__ding_text_alert_list" .Alerts.Resolved }}
  {{- end }}
{{- end }}

8. 告警效果

image.png

image.png

image.png