centos7安装配置filebeat7.9.1收集日志输出到windows kafka集群

下载filebeat（可以用迅雷或者wget）

https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.9.1-linux-x86_64.tar.gz

解压安装包并将其注册成服务（解决filebeat运行一段时间自动停止问题）

[root@wuhao ~]# tar -xf filebeat-7.9.1-linux-x86_64.tar.gz -C /usr/local/
[root@wuhao ~]# echo "110.110.110.110 WIN-KVL9BDV8B27" >>/etc/hosts  #Windows Server必须添加这一条解析。IP为windows服务器IP，主机名为windows服务器主机名，不然会出现识别不了主机的情况，Linux服务器则不需要添加
[root@wuhao ~]# vim /usr/lib/systemd/system/filebeat.service
[Unit]
Description=Filebeat is a lightweight shipper for metrics.
Documentation=https://www.elastic.co/products/beats/filebeat
Wants=network-online.target
After=network-online.target

[Service]
Environment="LOG_OPTS=-e"
Environment="CONFIG_OPTS=-c /usr/local/filebeat/filebeat.yml"
Environment="PATH_OPTS=-path.home /usr/local/filebeat -path.config /usr/local/filebeat -path.data /usr/local/filebeat/data -path.logs /usr/local/filebeat/logs"
ExecStart=/usr/local/filebeat/filebeat $LOG_OPTS $CONFIG_OPTS $PATH_OPTS
Restart=always

[Install]
WantedBy=multi-user.target

[root@wuhao ~]# chmod +x /usr/lib/systemd/system/filebeat.service

编辑配置filebeat

[root@wuhao ~]# vim /usr/local/filebeat/filebeat.yml

filebeat配置文件示例

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/gitee/supervisor_search.log   #收集单个日志
  tags: ["1"]                            #这里的tag是之后kibana检索相应日志的字段



- type: log
  enabled: true
  paths:
    - /var/gitee/*.log                   #收集以.log结尾的所有日志
  tags: ["2"]                            #自定义标签



- type: log
  enabled: true
  paths:
    - /var/gitee/*/*.log                 #收集以gitee子目录下一.log结尾的所有日志
  tags: ["3"]                            #自定义标签


output.kafka:
  enabled: true                    
  hosts: ["110.110.110.110:9092"]        #kafka服务器的ip和设置的端口，默认9092
  topic: 'kafka_run_log'                 #kafka创建的topic

检查配置文件是否有错

[root@wuhao ~]# cd /usr/local/filebeat/ && ./filebeat test config
Config OK       #   有这个提示表示没问题

启动filebeat

[root@wuhao ~]# systemctl daemon-reload
[root@wuhao ~]# systemctl enable filebeat
[root@wuhao ~]# systemctl start filebeat

查看filebeat运行状态

[root@wuhao filebeat]# systemctl status filebeat.service
● filebeat.service - Filebeat is a lightweight shipper for metrics.
   Loaded: loaded (/usr/lib/systemd/system/filebeat.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2020-08-21 17:50:37 CST; 9s ago      #有这一行，说明运行成功
     Docs: https://www.elastic.co/products/beats/filebeat
 Main PID: 6886 (filebeat)
   CGroup: /system.slice/filebeat.service
           └─6886 /usr/local/filebeat/filebeat -e -c /usr/local/filebeat/filebeat.y...

Aug 21 17:50:37 wuhao filebeat[6886]: 2020-08-21T17:50:37.145+0800        INFO   ...d.
Aug 21 17:50:37 wuhao filebeat[6886]: 2020-08-21T17:50:37.146+0800        INFO   ...og
Aug 21 17:50:37 wuhao filebeat[6886]: 2020-08-21T17:50:37.147+0800        INFO   ...og
Aug 21 17:50:37 wuhao filebeat[6886]: 2020-08-21T17:50:37.148+0800        INFO   ...og
Aug 21 17:50:37 wuhao filebeat[6886]: 2020-08-21T17:50:37.148+0800        INFO   ...og
Aug 21 17:50:37 wuhao filebeat[6886]: 2020-08-21T17:50:37.148+0800        INFO   ...og
Aug 21 17:50:38 wuhao filebeat[6886]: 2020-08-21T17:50:38.148+0800        INFO   ...er
Aug 21 17:50:38 wuhao filebeat[6886]: 2020-08-21T17:50:38.148+0800        INFO   ...2)
Aug 21 17:50:38 wuhao filebeat[6886]: 2020-08-21T17:50:38.148+0800        INFO   ...ne
Aug 21 17:50:38 wuhao filebeat[6886]: 2020-08-21T17:50:38.148+0800        INFO   ...ed
Hint: Some lines were ellipsized, use -l to show in full.

配置完成