prometheus-operator 之企业邮箱告警

1.1.1.5 内网服务器 k8s master(修改默认告警系统)
1.1.1.1 外部机器(搭建邮件转发nginx)

1. nginx搭建

cat nginx.conf

stream{

    server{
        listen 4650;
        proxy_pass exmail.xxx.com:587;

          }
      }

---

2. 告警系统

此配置只适用企业邮箱，qq网易邮箱得自行网上寻找配置

2.1 alertmanager.yaml

mkdir -p /home/kubernetes/k8s/app/alertmanager_alert
cd /home/kubernetes/k8s/app/alertmanager_alert
touch alertmanager.yaml

#cat alertmanager.yaml
global:
  resolve_timeout: 5m
  smtp_smarthost: '1.1.1.1:4650'
  smtp_from: '[email protected]'
  smtp_auth_username: '[email protected]'
  smtp_auth_password: 'password'
  smtp_hello: 'xxx.com'
  smtp_require_tls: true

templates:
  - "./*.tmpl"

receivers:
- name: "email"
  email_configs:
  - to: '[email protected]'
    html: '{{ template "email.html" . }}'
    send_resolved: true
    tls_config:
      insecure_skip_verify: true

route:
  group_by:
  - job
  group_interval: 30s
  group_wait: 5s
  receiver: "email"
  repeat_interval: 12h

2.2 新增告警模板

cd /home/kubernetes/k8s/app/alertmanager_alert
mkdir template
touch template/email.tmpl

#cat touch template/email.tmpl
#抄的 http://www.manongjc.com/detail/26-ysorykovlufnizt.html(http://www.manongjc.com/detail/26-ysorykovlufnizt.html)


{{ define "email.html" }}
{{- if gt (len .Alerts.Firing) 0 -}}
{{- range $index, $alert := .Alerts -}}

=========  监控告警 =========


 告警程序:     Alertmanager 

 告警类型:    {{ $alert.Labels.alertname }} 

 告警级别:    {{ $alert.Labels.severity }} 级 

 告警状态:    {{ .Status }} 

 故障主机:    {{ $alert.Labels.instance }} {{ $alert.Labels.device }} 

 告警主题:    {{ .Annotations.summary }} 

 告警详情:    {{ $alert.Annotations.message }}{{ $alert.Annotations.description}} 

 主机标签:    {{ range .Labels.SortedPairs  }} 
 [{{ .Name }}: {{ .Value  | html }} ]{{ end }}

 故障时间:    {{ ($alert.StartsAt.Add 28800e9).Format "2006-01-02 15:04:05" }}


========= = end =  =========











    

        
xxxx
        

        (这是一封自动发送的邮件，请勿回复。)
        
    


    

        

            xxx
        
        
            {{ ($alert.StartsAt.Add 28800e9).Format "2006-01-02 " }}

        
    



{{- end }}
{{- end }}
{{- if gt (len .Alerts.Resolved) 0 -}}
{{- range $index, $alert := .Alerts -}}
=========  告警恢复 =========

 告警程序:    Alertmanager 

 告警主题:    {{ $alert.Annotations.summary }}

 告警主机:    {{ .Labels.instance }} 

 告警类型:    {{ .Labels.alertname }}

 告警级别:    {{ $alert.Labels.severity }} 级 

 告警状态:    {{   .Status }}

 告警详情:    {{ $alert.Annotations.message }}{{ $alert.Annotations.description}}

 故障时间:    {{ ($alert.StartsAt.Add 28800e9).Format "2006-01-02 15:04:05" }}

 恢复时间:    {{ ($alert.EndsAt.Add 28800e9).Format "2006-01-02 15:04:05" }}

========= = end =  =========










    

        
xxxx
        

        (这是一封自动发送的邮件，请勿回复。)
        
    


    

        

            xxx
        
        
            {{ ($alert.StartsAt.Add 28800e9).Format "2006-01-02 " }}

        
    

{{- end }}
{{- end }}
{{- end }}

---

2.3 删除默认配置，换上新配置

cd /home/kubernetes/k8s/app/alertmanager_alert
kubectl delete -n monitoring secrets alertmanager-main
kubectl  create   secret generic  alertmanager-main --from-file=alertmanager.yaml --from-file=./template/email.tmpl -n monitoring