Filebeat+Logstash+Elasticsearch+Kibana实现图形化日志监控

Elasticsearch集群启动

启动es集群

Filebeat 配置及启动

创建配置文件log-monitor.yml内容如下：

[root@localhost filebeat]# vi log-monitor.yml
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /opt/apps/filebeat/logs/*.log
setup.template.settings:
  index.number_of_shards: 3
output.logstash:
  hosts: ["192.168.12.10:5044"]```

启动 filebeat服务
[root@localhost filebeat]# ./filebeat -e -c log-monitor.yml

Logstash 配置及启动

创建配置文件log-moni.conf内容如下：

[root@localhost logstash]# vi log-moni.conf
input {
 beats {
  port => "5044"
 }
}
filter {
 mutate {
  split => {"message"=>"|"}
 }
 mutate {
  add_field => {
    "userId" => "%{message[1]}"
    "visit" => "%{message[2]}"
    "date" => "%{message[3]}"
  }
 }
 mutate {
  convert => {
    "userId" => "integer"
    "visit" => "string"
    "date" => "string"
  }
 }
}
output {
 elasticsearch {
   hosts => ["192.168.12.10:9200","192.168.12.11:9200","192.168.12.12:9200"]
 }
}

启动Logstash服务

[root@localhost logstash]# ./bin/logstash -f log-moni.conf

模拟写入数据到日志文件

[root@localhost logs]# vi moni.log
[INFO] 2019-03-15 22:54:42 [cn.itcast.dashboard.Main] - DAU|4645|领取优惠券|2019-03-15 07:40:29
[INFO] 2019-03-15 22:54:44 [cn.itcast.dashboard.Main] - DAU|3482|领取优惠券|2019-03-15 18:34:04
[INFO] 2019-03-15 22:54:48 [cn.itcast.dashboard.Main] - DAU|5607|加入收藏|2019-03-15 22:44:09
[INFO] 2019-03-15 22:54:50 [cn.itcast.dashboard.Main] - DAU|9619|加入收藏|2019-03-15 21:39:47
[INFO] 2019-03-15 22:54:53 [cn.itcast.dashboard.Main] - DAU|7666|加入收藏|2019-03-15 17:47:18
[INFO] 2019-03-15 22:54:54 [cn.itcast.dashboard.Main] - DAU|4871|提交订单|2019-03-15 02:36:27
[INFO] 2019-03-15 22:54:55 [cn.itcast.dashboard.Main] - DAU|7126|加入收藏|2019-03-15 16:11:06
[INFO] 2019-03-15 22:55:00 [cn.itcast.dashboard.Main] - DAU|9606|评论商品|2019-03-15 02:12:00
[INFO] 2019-03-15 22:55:02 [cn.itcast.dashboard.Main] - DAU|7698|查看订单|2019-03-15 08:17:01

可以看到数据同步到ES中

Kibana启动及配置

启动 Kibana
./bin/kibana
添加Logstash索引

discover 界面可以看到数据

创建自定义图表条形图并保存

创建自定义图表 饼图并保存

创建自定义dashboard 并添加我们自定义的图表

模拟产生日志数据可以看到图表实时变化