notice - - - {"kill_chain":"remote-control","dst_ip_geoloc":"34.7725,113.7266","sub_category":"APT_htran (apt );APT_others (apt );APT_others(apt )","src_ip_city":"Beijing","hostip":"10.1.0.37","reliability":10,"classtype":"threat-intelligence-alarm","increment":0,"dst_ip_country_code":"CN","event_source":"NDE","rid":"10180001","interface":"zc:99@2","uuid":"2ecf0bff-6eb1-42a7-8f0b-ff6eb1a2a786","dst_ip":"114.114.114.114","src_ip":"10.1.0.198","dst_service":"dns","hostname":"ATD","commid":"ECtmCxaV8nDaCsMKjHknBj41QEo=","sensor_ip":"10.1.0.37","timestamp":"1550799035951","severity":"5","dns":{"qtype_name":"A","query":"bbs.india-videoer.com"},"dst_ip_country":"China","message":"DNS resolves malicious domain names","app_proto":"dns","target":"bbs.india-videoer.com","src_port":"51095","engine_type":"intel","src_ip_country_code":"CN","proto":"udp","dst_port":"53","src_ip_geoloc":"39.9047,116.4072","src_ip_country":"China","family":"APT_htran (apt );APT_others (apt );APT_others(apt )","category":"malicious-domain-dns-query","desc":"family:APT_htran (apt )APT_others (apt )APT_others(apt );query:bbs.india-videoer.com;answers:;qtype_name:A;","sensor_name":"ATD"}
需要将以上数据源格式化为json存入es中,首先需要将
notice - - -
替换为空字符串,然后在filter中配置自动解析json格式
input {
stdin {}
}
filter{
mutate {
gsub =>[
"message", "notice - - - ", ""
]
}
json{
source => "message"
}
}
output {
stdout {}
}
logstash会自动见json中的键解析为es中的字段进行存储