grafana报错: context deadline exceeded

版本 grafana:5.4.5 docker镜像
系统 centos7.4

发现钉钉收不到grafana报警,查看日志发现报错如下

{"log":"t=2020-06-01T15:33:34+0800 lvl=eror msg=\"Failed to send DingDing\" logger=alerting.notifier.dingding error=\"context deadline exceeded\" dingding=系统钉钉报警\n","stream":"stdout","time":"2020-06-01T07:33:34.433482572Z"}
{"log":"t=2020-06-01T15:33:34+0800 lvl=eror msg=\"failed to send notification\" logger=alerting.notifier id=1 error=\"context deadline exceeded\"\n","stream":"stdout","time":"2020-06-01T07:33:34.433491039Z"}
{"log":"t=2020-06-01T15:40:12+0800 lvl=info msg=\"New state change\" logger=alerting.resultHandler alertId=50 newState=alerting prev state=ok\n","stream":"stdout","time":"2020-06-01T07:40:12.473003766Z"}

重启grafana镜像会短暂恢复正常,几个小时后又会出现无法报警的问题

目前参考github上的解决方案已经恢复正常报警

首先安装grafana-image-renderer插件

grafana-cli plugins install grafana-image-renderer

安装完成后将grafana.ini配置文件改为如下

#################################### Alerting ############################
[alerting]
concurrent_render_limit = 15
evaluation_timeout_seconds = 60
notification_timeout_seconds = 60
max_attempts = 5

然后重启grafana服务

参考链接:https://github.com/grafana/grafana/issues/21256

你可能感兴趣的:(linux,grafana,运维)