logstash中grok匹配

关于logstash中grok匹配的一点经验:

#这个网站中正则表达的质量比较高:https://github.com/yangzj1992/articles/blob/master/%E3%80%90%E4%B8%AA%E4%BA%BA%E6%80%BB%E7%BB%93%E3%80%91%E6%AD%A3%E5%88%99%E8%A1%A8%E8%BE%BE%E5%BC%8F%E8%AF%AD%E6%B3%95%E5%8F%8A%E5%B8%B8%E7%94%A8%E6%AD%A3%E5%88%99.md

input {

file {

path => "c:\Program Files\logstash-2.3.3\jifeng_test1.txt"

}

}

filter {

grok {

match => { "message" => "%{USERNAME_CHINESE_DONG:username}\~\|\^%{EMAILADDRESS_DONG:email}" }

#关于这一行有几点要注意:每条日志的样子为:  anetadmin~|^[email protected]~|^3e46cea508a2595dc54510a10d5bbb1e:203473~|^222.130.140.89

#我们用match去匹配时不能写成:

# match => { "message" => "%{USERNAME_CHINESE_DONG:username}~|^%{EMAILADDRESS_DONG:email}" }

#因为 "|" 和 "^"是正则表达式中的特殊字符,需要用"\"转义,就像第10行所示.另外,"~"不是正则表达式的特殊字符,

#但我测试过,在"~"前加"\"也可以正确运行.即以下match匹配都对:

# match => { "message" => "%{USERNAME_CHINESE_DONG:username}\~\|\^%{EMAILADDRESS_DONG:email}" }

# match => { "message" => "%{USERNAME_CHINESE_DONG:username}~\|\^%{EMAILADDRESS_DONG:email}" }

#另外有一点:系统中自带的email匹配EMAILADDRESS输出结果不正确,所以我是用了自定义的EMAILADDRESS_DONG去匹配.

#EMAILADDRESS_DONG的定义位于C:\Program Files\logstash-2.3.3\vendor\bundle\jruby\1.9\gems\logstash-patterns-core-2.0.5\patterns

#match中正则表达式的部分是有严格的格式的,不能随便加入空格\制表符等东西.

}

}

output{

stdout{ codec => rubydebug{}}

#elasticsearch {

# flush_size => 10000

# hosts => "localhost"

# index => "qq_sgk"

# document_type => "qq_sgk-1"

#}

}