ELK-kibana来源热点图

环境
- ELK5.6
- centos6.9

官方文档
- geoip

elasticsearch需要的插件
./elasticsearch-plugin install ingest-geoip

- 重启ES，插件生效
下载ip库

wget http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz

• 解压

gunzip GeoLite2-City.mmdb.gz

• 生成文件：

-rw-r--r--  1 root root  61M Jan  5 23:10 GeoLite2-City.mmdb

将其放在相应目录
logstash日志格式-json

{ "@timestamp": "2018-02-05T00:00:54+08:00", "hostname": "dny-l7-x-x", "server_name": "www.xxxxx.com", "http_x_forwarded_for": "xxx.xxx.xxx.xxx", "xxx-app": "xxx-app/8141", "remote_addr": "xxx.xxx.xxx.xxx", "remote_user": "-", "body_bytes_sent": 11870, "request_time": 0.002, "upstream_response_time": "0.002", "status": 200, "upstream_status": "200", "connection_requests": 1, "request": "GET /xxx/shiti/2112581.html HTTP/1.1", "request_method": "GET", "request_body": "-", "http_referrer": "-", "http_cookie": "-", "http_user_agent": "Mozilla/5.0 (Linux;u;Android 4.2.2;zh-cn;) AppleWebKit/534.46 (KHTML,like Gecko) Version/5.1 Mobile Safari/10600.6.3 (compatible; Baiduspider/2.0;+http://www.baidu.com/search/spider.html)"}


在nginx中的配置

http {
.........
log_format main '{ "@timestamp": "$time_iso8601", '
'"hostname": "$hostname", '
'"server_name": "$server_name", '
'"http_x_forwarded_for": "$http_x_forwarded_for", '
'"xxx-app": "$upstream_http_server", '
'"remote_addr": "$remote_addr", '
'"remote_user": "$remote_user", '
'"body_bytes_sent": $body_bytes_sent, '
'"request_time": $request_time, '
'"upstream_response_time":"$upstream_response_time", '
'"status": $status, '
'"upstream_status": "$upstream_status", '
'"connection_requests": $connection_requests, '
'"request": "$request", '
'"request_method": "$request_method", '
'"request_body": "$request_body", '
'"http_referrer": "$http_referer", '
'"http_cookie": "$http_cookie", '
'"http_user_agent": "$http_user_agent" } ';
server {
..............
access_log logs/access_json.log main;
...........
}
........
}

logstash文件

/etc/logstash/conf.d/logstah_agent.conf
input {
   file {
       path => "/usr/local/lnmp/tengine/logs/access.log"
       start_position => beginning
       codec => "json"
    }
}
filter {
    geoip {
        source => "http_x_forwarded_for"
        # 过滤的字段，获得IP地址
        target => "geoip"
        database => "/usr/share/kibana/GeoLite2-City.mmdb"
        # 解压的IP库文件
        add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
        add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}"  ]
        # 经纬度
    }
}
output {
    elasticsearch {
    hosts => "10.99.2.17:9200"
    }
    stdout { codec => rubydebug }
}

---

然后启动ELK三个软件

$ /usr/share/elasticsearch/bin/elasticsearch -d
# /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash_agent.conf
# /usr/share/kibana/bin/kibana

• 在浏览器中输入 http://10.99.2.17:5601 进入kibana页面:
• 选择Visualize->Maps->Coordinate Map
• 选择索引logstash-*
• 配置界面：
  
• 效果图：
  
• 在logstash中的内容：
  

如果出现
“_geoip_lookup_failure“问题，可能是你的IP库没添加，配置文件中”source”解析的字段问题，或者是IP不是公网IP！