CentOS 6.5 x64安装ELK日志分析系统
CentOS 6.5 x64安装ELK日志分析系统
环境说明：
系统是CentOS 6.5x64,已经安装好编译环境
已经安装好Tengine2.10
已经关闭iptables
核心软件包如下:
elasticsearch-1.4.2.tar.gz
下载链接:
https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.4.2.tar.gz
logstash-1.4.2.tar.gz
下载链接:
https://download.elasticsearch.org/logstash/logstash/logstash-1.4.2.tar.gz
kibana-3.1.2.tar.gz
下载链接:
https://download.elasticsearch.org/kibana/kibana/kibana-3.1.2.tar.gz
特别声明一点elasticsearch和kibana有版本兼容问题，很多人，网页死都出不来，就是因为这个原因。
所以，如果要参考这篇文章，请严格按照版本来,不要下载最新版。
安装JDK
ElasticSearch和Logstash依赖于JDK
tar zxvfjdk-7u71-linux-x64.tar.gz
mv jdk1.7.0_71/usr/local/java
vi /etc/profile
最后一行添加
#Java environment
exportJAVA_HOME=/usr/local/java
exportPATH=$PATH:$JAVA_HOME/bin
exportCLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib:$JAVA_HOME/bin
加载环境变量
source /etc/profile
查看版本
[root@localhost ~]#java -version
java version"1.7.0_45"
OpenJDK RuntimeEnvironment (rhel-2.4.3.3.el6-x86_64 u45-b15)
OpenJDK 64-Bit ServerVM (build 24.45-b08, mixed mode)
安装ElasticSearch
ElasticSearch默认的对外服务的HTTP端口是9200，节点间交互的TCP端口是9300
tar zxvfelasticsearch-1.4.2.tar.gz
mv elasticsearch-1.4.2/usr/local/elasticsearch
修改ElasticSearch的配置文件，追加一行内容,否则Kibana页面提示Connection Failed
echo"http.cors.enabled: true" >>/usr/local/elasticsearch/config/elasticsearch.yml
安装elasticsearch-servicewrapper
wgethttps://github.com/elasticsearch/elasticsearch-servicewrapper/archive/master.tar.gz
tar zxvf master.tar.gz
cp -relasticsearch-servicewrapper-master/service /usr/local/elasticsearch/bin/
启动ElasticSearch服务
/usr/local/elasticsearch/bin/service/elasticsearchstart
等待5秒，查看端口
[root@localhost jar]# netstat -anpt | grep 9200
tcp 0 0 :::9200 :::* LISTEN 15682/java
测试ElasticSearch服务是否正常，预期返回200的状态码：
[root@localhost ~]# curl -X GET http://localhost:9200
{
"status": 200,
"name" : "Whistler",
"cluster_name" :"elasticsearch",
"version" : {
"number" : "1.4.2",
"build_hash" :"927caff6f05403e936c20bf4529f144f0c89fd8c",
"build_timestamp" :"2014-12-16T14:11:12Z",
"build_snapshot" : false,
"lucene_version" :"4.10.2"
},
"tagline" : "You Know, forSearch"
}
安装Logstash
Logstash默认的对外服务的端口是9292。
tar zxvflogstash-1.4.2.tar.gz
mv logstash-1.4.2/usr/local/logstash
mkdir -p/usr/local/logstash/etc
创建Logstash配置文件logstash_agent.conf，这里将Nginx日志和messages系统日志作为输入，输出直接传给ElasticSearch
里面的IP地址是服务器的IP地址
vim /usr/local/logstash/etc/logstash_agent.conf
input {
file {
type => "nginx.access"
path =>["/usr/local/nginx/logs/access.log"]
}
file {
type => "nginx.error"
path =>["/usr/local/nginx/logs/error.log"]
}
output {
elasticsearch {
host => "192.168.1.105"
port => 9300
}
}
启动logstash
/usr/local/logstash/bin/logstash-f /usr/local/logstash/etc/logstash_agent.conf &
编辑index配置文件，主要负责解析日志格式的
vi logstash_indexer.conf
将filter部分删除，添加新的filter。效果如下：
input {
file {
type => "nginx.access"
path => ["/usr/local/nginx/logs/access.log"]
}
}
filter {
grok {
type => "nginx.access"
match => [
"message", "%{IPORHOST:source_ip} - %{USERNAME:remote_user} \[%{HTTPDATE:timestamp}\] %{QS:request} %{INT:status} %{INT:body_bytes_sent} %{QS:http_referer} %{QS:http_user_agent}"
]
}
}
output {
elasticsearch {
embedded => false
protocol => "http"
host => "192.168.1.105"
port => "9200"
}
}
启动index配置文件
/usr/local/logstash/bin/logstash-f /usr/local/logstash/etc/logstash_indexer.conf &
安装Kibana
Kibana的源代码就是一个网页，所以直接用Nginx发布就可以了。
mkdir /www
tar zxvfkibana-3.1.2.tar.gz
mv kibana-3.1.2/www/kibana
修改Kibana的配置文件，指定elasticsearch
vim /www/kibana/config.js
32 elasticsearch:"http://"+window.location.hostname+":9200",
修改为:
elasticsearch:"http://192.168.1.105:9200",
修改Nginx配置文件
vim/usr/local/nginx/conf/vhosts/kibana.conf
server {
listen 80;
server_name kibana.xx.com;
root /www/kibana;
index index.php index.html index.htm;
location / {
}
location ~ \.php$ {
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi_params;
}
}
重新加载nginx配置
/usr/local/nginx/sbin/nginx-s reload
访问页面
http://kibana.xx.com/
提示LogStash仪表板设置
点击右边的样品仪表板
进入页面
在下面就可以看到Nginx日志了
点击source_ip,选择bar
可以看到排名前十的客户端访问IP地址
其他的功能可以自己慢慢看
本文参考
http://blog.csdn.net/i_chips/article/details/43309415