ELK之Filebeat安装与配置及使用

1 Filebeat介绍

轻量型日志采集器。当您要面对成百上千、甚至成千上万的服务器、虚拟机和容器生成的日志时，请告别 SSH 吧。Filebeat 将为您提供一种轻量型方法，用于转发和汇总日志与文件，让简单的事情不再繁杂。

2 安装

# wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.0.0-linux-x86_64.tar.gz
# tar zxf filebeat-6.0.0-linux-x86_64.tar.gz -C /Data/apps/

3 应用一：Filebeat采集数据，写入到logstash

3.1 修改Filebeat配置

101.1.21.170是logstash机器的ip。

# cd /Data/apps/filebeat-6.0.0/
# sudo vi filebeat.yml

filebeat.prospectors:
enabled: true
-  /Data/apps/nginx/logs/access-filebeat-test.log
output.logstash:
hosts: ["101.1.21.170:5044"]

启动Filebeat

# sudo ./filebeat -e -c filebeat.yml

3.2 logstash配置

101.1.21.170是logstash机器的ip。

# /Data/apps/logstash-6.0.0/bin
# sudo vi filebeat.conf


input {
    beats {
        port => 5044
    }
}


output {
    stdout {
        codec => "rubydebug"
    }
}

启动logstash

# ./logstash -f filebeat.conf

3.3 向/Data/apps/nginx/logs/access-filebeat-test.log追加信息

# echo "CleverCode is coding" >> /Data/apps/nginx/logs/access-filebeat-test.log

4 应用二：Filebeat采集数据，写入到redis，然后logstash从reids队列中拉取数据

4.1 filebeat配置

# cd /Data/apps/filebeat-6.0.0/
# sudo vi filebeat.yml
filebeat.prospectors:
	 enabled: true
	 -  /Data/apps/nginx/logs/access-filebeat-test.log
output.redis:
     hosts: ["10.1.20.170"]
     port: 6379
     key: "filebeat"
     db: 0
     timeout: 5
	 

启动Filebeat

# sudo ./filebeat -e -c filebeat.yml

4.2 logstash配置

# /Data/apps/logstash-6.0.0/bin
# sudo vi redis.conf
input {
    redis {
        host => "10.1.20.170"
        port => 6379
        key => "filebeat"
        data_type => "list"
    }
}
output {
    stdout {
        codec => rubydebug
    }
}

启动logstash

# ./logstash -f redis.conf

4.3 向/Data/apps/nginx/logs/access-filebeat-test.log追加信息

# echo "this msg is from reids" >> /Data/apps/nginx/logs/access-filebeat-test.log