Logstash安装

环境

系统：CentOS 6.9
软件：logstash-6.1.0.tar.gz

准备

• 安装JDK
  参见：http://www.jianshu.com/p/9d1b3eefdea3

安装

• 下载二进制包
  地址：https://www.elastic.co/downloads/logstash

• 创建用户

  # useradd elk
  

• 安装

  # tar -xzvf logstash-6.1.0.tar.gz
  # mv logstash-6.1.0 /usr/local/logstash
  # chown -R elk:elk /usr/local/logstash
  

配置

• 自定义匹配模式

  $ su - elk
  $ mkdir /usr/local/logstash/patterns
  $ vim /usr/local/logstash/patterns/tomcat
  TOMCAT_DATETIME %{MONTHDAY}-%{MONTH}-%{YEAR} %{TIME}
  

• 编辑配置文件

$ vim /usr/local/logstash/logstash.conf
input {
  kafka {
    bootstrap_servers => "elk-kafka-1:9092"
    topics => ["filebeat"]
    codec => "json"
  }
}

filter {
  if [log_format] == "datatime-level-thread-class-content" {
    grok {
      patterns_dir => ["./patterns"]
      match => {"message" => "%{TOMCAT_DATETIME:datetime} %{LOGLEVEL:level} \[(?\w+)\] %{JAVACLASS:class} %{GREEDYDATA:content}"}
    }
    date {
      match => [ "datetime" , "dd-MMM-yyyy HH:mm:ss.SSS" ]
    }
  }
}

output {
  elasticsearch {
    hosts => [ "elk-elasticsearch-1:9200", "elk-elasticsearch-2:9200" ]
    index => "%{[project]}-%{+YYYY.MM.dd}"
  }
}

启动

• 开机启动

  # vim /etc/rc.local
  su admin -c 'cd /usr/local/logstash && nohup ./bin/logstash -f logstash.conf >> /tmp/logstash.log 2>&1 &'
  

• 启动

  # su - elk
  $ cd /usr/local/logstash
  $ nohup ./bin/logstash -f logstash.conf >> /tmp/logstash.log 2>&1 &