docker-elk配置sentinl插件发送钉钉告警

安装elk

下载elk docker工具包并启动 elk 6.6.1

https://github.com/deviantony/docker-elk.git
cd docker-elk && git checkout 366e2bc

修改docker-compose.yml挂载kibana插件目录

vim docker-compose.yml

kibana:
  build:
    context: kibana/
    args:
      ELK_VERSION: $ELK_VERSION
  volumes:
    - ./kibana/config/:/usr/share/kibana/config:ro
    - ./kibana/plugins:/usr/share/kibana/plugins
  ports:
    - "5601:5601"
  networks:
    - elk
  depends_on:
    - elasticsearch

启动elk

docker-compose up -d

安装sentinl

进入kibana容器内部安装sentinl 6.6.1

/opt/kibana/bin/kibana-plugin install https://github.com/sirensolutions/sentinl/releases/download/tag-6.6.0-0/sentinl-v6.6.1.zip

重启elk
docker-compose down && docker-compose up

image

添加watcher

image

选择Watcher Advanced

image

在钉钉pc客户端里生成机器人webhook url并复制

image

输入监控配置文件内容

{
  "actions": {
    "Webhook_683bd385-86b3-46ba-8e1b-f89cccccbbec": {
      "name": "error异常告警",
      "throttle_period": "1m",
      "webhook": {
        "priority": "high",
        "stateless": false,
        "method": "POST",
        "host": "oapi.dingtalk.com",
        "port": "443",
        "path": "/robot/send?access_token=你的钉钉token",
        "body": "{\"msgtype\": \"text\", \"text\": {\"content\":\"index：{{payload.hits.hits.0._index}}\nsource：{{payload.hits.hits.0._source.source}}\nenv：{{payload.hits.hits.0._source.env}}\nthread: {{payload.hits.hits.0._source.thread}}\nmessage:{{payload.hits.hits.0._source.message}}\ntimes:{{payload.hits.total}}\nthrowable: {{payload.hits.hits.0._source.throwable}}\"}}",
        "params": {
          "watcher": "{{watcher.title}}",
          "payload_count": "{{payload.hits.total}}"
        },
        "headers": {
          "Content-Type": "application/json"
        },
        "message": "业务功能告警",
        "use_https": true
      }
    }
  },
  "input": {
    "search": {
      "request": {
        "index": [
          "logstash-*"
        ],
        "body": {
          "query": {
            "bool": {
              "must": [
                {
                  "match": {
                    "level": "ERROR"
                  }
                },
                {
                  "range": {
                    "@timestamp": {
                      "gte": "now-5m",
                      "lte": "now",
                      "format": "epoch_millis"
                    }
                  }
                }
              ],
              "must_not": []
            }
          }
        }
      }
    }
  },
  "condition": {
    "script": {
      "script": "payload.hits.total >=1"
    }
  },
  "trigger": {
    "schedule": {
      "later": "every 5 minutes"
    }
  },
  "disable": true,
  "report": false,
  "title": "钉钉告警",
  "save_payload": false,
  "spy": true,
  "impersonate": false
}

查看效果

image

sentinl 支持多种通知渠道，可根据需求选择

image

多种elk报警插件可供选择

• https://github.com/Yelp/elastalert
• http://docs.flycloud.me/docs/ELKStack/elasticsearch/other/watcher.html
• https://blog.52itstyle.vip/archives/3137/

可根据需求灵活选择