elasticsearch+filebeat日志分析

起先使用logback+logstash将数据收录到日志仓库中，但是考虑到该方案会消耗java运行内存（如果日志数据量太大可能会影响系统的正常运行）

  最后在官网找到了一个可以分析日志文件的日志收集器并收录到elasticsearch仓库中 (filebeat )

1:安装filebeat,有三种方式

deb:
curl -L -O https://download.elastic.co/beats/filebeat/filebeat_1.2.3_amd64.deb
sudo dpkg -i filebeat_1.2.3_amd64.deb

rpm:
curl -L -O https://download.elastic.co/beats/filebeat/filebeat-1.2.3-x86_64.rpm
sudo rpm -vi filebeat-1.2.3-x86_64.rpm

mac:
curl -L -O https://download.elastic.co/beats/filebeat/filebeat-1.2.3-darwin.tgz
tar xzvf filebeat-1.2.3-darwin.tgz

2:配置filebeat（撇弃logstash,直接将数据扔给es）

output:
  elasticsearch:
    hosts: ["localhost:9200"]
    # A template is used to set the mapping in Elasticsearch
    # By default template loading is disabled and no template is loaded.
    # These settings can be adjusted to load your own template or overwrite existing ones
    template:
      # Template name. By default the template name is filebeat.
      #name: "filebeat"
      # Path to template file
      path: "filebeat.template.json"
      # Overwrite existing template
      #overwrite: false

创建/加载模板

deb or rpm:
curl -XPUT 'http://localhost:9200/_template/filebeat' -d@/etc/filebeat/filebeat.template.json

mac:
cd filebeat-1.2.3-darwin
curl -XPUT 'http://localhost:9200/_template/filebeat' [email protected]

win:
PS C:\Program Files\Filebeat> Invoke-WebRequest -Method Put -InFile filebeat.template.json -Uri http://localhost:9200/_template/filebeat?pretty

启动filebeat

deb:
sudo /etc/init.d/filebeat start

rpm:
sudo /etc/init.d/filebeat start

mac:
sudo ./filebeat -e -c filebeat.yml -d "publish"

win:

PS C:\Program Files\Filebeat> Start-Service filebeat

完成,有点乱，将就看吧 elk详细搭建我博客里有,可以参照一下

curl -XDELETE http://192.168.161.92:9200/logstash-2016.07.*   删除es索引