Contents [hide]
1 基本介绍
2 安装过程
2.1 准备
2.2 安装java
2.3 Elasticsearch
2.4 kibana
2.5 Logstash
2.6 Logstash Forwarder
3 增加节点
4 参考资料
基本介绍
Elasticsearch 目前最新的版本是1.7.1 ，
Logstash 目前最新版本是1.5.3
kibana 目前最新版本是：4.1.1
Logstash Forwarder，目前最新的版本是0.4.0
安装过程
创建一个虚拟机CentOS7.1，分配公网IP，来进行测试
准备
查看操作系统版本
cat /etc/redhat-release CentOS Linux release 7.1.1503 (Core)
查看Selinux状态
sestatus SELinux status: disabled
FQDN
设置FQDN，其实有两种方法，http://www.opsers.org/base/linux-fqdn-set.html 这位朋友总结的很好。
记住：创建SSL证书的时候需要用到，这是必做
cat /etc/hostname #i-2053f7hj elk
# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 # hostname loopback address 127.0.1.1 i-2053f7hj 10.70.26.187 elk.chenshake.com elk
然后运行
hostname -F /etc/hostname
这时候你可以查看自己的FQDN名字
# hostname -f elk.chenshake.com # hostname elk
安装Firewall
CentOS7有一个重大改变，用firewalld取代iptables，大家都习惯iptables，所以×××默认的镜像，就是使用iptables，没有安装firewalld。
yum install firewalld firewall-config systemctl start firewalld.service systemctl enable firewalld.service systemctl status firewalld
查看是否生效
firewall-cmd --state
安装java
yum install java-1.7.0-openjdk
Elasticsearch
wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.1.noarch.rpm
安装
yum localinstall elasticsearch-1.7.1.noarch.rpm
启动相关服务
照提示做就可以。
systemctl daemon-reload systemctl enable elasticsearch.service systemctl start elasticsearch systemctl status elasticsearch
查看Elasticsearch的配置文件
# rpm -qc elasticsearch /etc/elasticsearch/elasticsearch.yml /etc/elasticsearch/logging.yml /etc/init.d/elasticsearch /etc/sysconfig/elasticsearch /usr/lib/sysctl.d/elasticsearch.conf /usr/lib/systemd/system/elasticsearch.service /usr/lib/tmpfiles.d/elasticsearch.conf
查看端口使用情况
# netstat -nltp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 606/sshd tcp6 0 0 :::9200 :::* LISTEN 3051/java tcp6 0 0 :::9300 :::* LISTEN 3051/java tcp6 0 0 :::22 :::* LISTEN 606/sshd
在使用9200和9300端口，防火墙打开端口
firewall-cmd --permanent --add-port={9200/tcp,9300/tcp}
firewall-cmd --reload
查看防火墙端口情况
# firewall-cmd --list-all public (default, active) interfaces: eth0 sources: services: dhcpv6-client ssh ports: 9200/tcp 9300/tcp masquerade: no forward-ports: icmp-blocks: rich rules:
kibana
下载tar包
wget https://download.elastic.co/kibana/kibana/kibana-4.1.1-linux-x64.tar.gz
解压
tar zxf kibana-4.1.1-linux-x64.tar.gz -C /usr/local/ cd /usr/local/ mv kibana-4.1.1-linux-x64 kibana
创建启动文件
创建一个kibana.service文件。
cat > /etc/systemd/system/kibana.service <启动kibana服务
systemctl enable kibana systemctl start kibana systemctl status kibana查看端口
# netstat -nltp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:5601 0.0.0.0:* LISTEN 3830/node tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 977/sshd tcp6 0 0 :::9200 :::* LISTEN 978/java tcp6 0 0 :::9300 :::* LISTEN 978/java tcp6 0 0 :::22 :::* LISTEN 977/sshd操作过程基本一样，打开5601端口
firewall-cmd --permanent --add-port=5601/tcp firewall-cmd --reload firewall-cmd --list-all这个时候，访问虚拟机，就可以通过 http://ip:5601端口访问。对于×××的系统，你需要在防火墙打开5601端口。
firewalld功能还是很强大，你可以设置转发
firewall-cmd --permanent --add-forward-port=port=80:proto=tcp:toport=5601 firewall-cmd --reload firewall-cmd --list-all
Logstash
下载rpm包
wget https://download.elastic.co/logstash/logstash/packages/centos/logstash-1.5.3-1.noarch.rpm安装
yum localinstall logstash-1.5.3-1.noarch.rpm设置ssl
这是最容易出错的地方，SSL证书创建，可以通过FQDN或者IP进行创建，我通过IP创建没有成功。我这里使用的FQDN名字是 elk.chenshake.com ,
cd /etc/pki/tls openssl req -subj '/CN=elk.chenshake.com/' -x509 -days 3650 -batch -nodes -newkey rsa:2048 -keyout private/logstash-forwarder.key -out certs/logstash-forwarder.crt
创建一个01-logstash-initial.conf 文件
cat > /etc/logstash/conf.d/01-logstash-initial.conf << EOF input { lumberjack { port => 5000 type => "logs" ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt" ssl_key => "/etc/pki/tls/private/logstash-forwarder.key" } } filter { if [type] == "syslog" { grok { match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" } add_field => [ "received_at", "%{@timestamp}" ] add_field => [ "received_from", "%{host}" ] } syslog_pri { } date { match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ] } } } output { elasticsearch { host => localhost } stdout { codec => rubydebug } } EOF启动logstash服务
systemctl start logstash systemctl status logstash查看端口，需要等2分钟，才能看到5000端口启动
# netstat -nltp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:5601 0.0.0.0:* LISTEN 3830/node tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 977/sshd tcp6 0 0 :::5000 :::* LISTEN 20017/java tcp6 0 0 :::9200 :::* LISTEN 978/java tcp6 0 0 :::9300 :::* LISTEN 978/java tcp6 0 0 :::9301 :::* LISTEN 20017/java tcp6 0 0 :::22 :::* LISTEN 977/sshdlogstash在监听5000端口
firewall-cmd --permanent --add-port=5000/tcp firewall-cmd --reload firewall-cmd --list-all
如果启动有问题，你可以去看logstash的日志/var/log/logstash
Logstash Forwarder
ELK的机器，也作为一个一个客户端进行收集日志
wget https://download.elastic.co/logstash-forwarder/binaries/logstash-forwarder-0.4.0-1.x86_64.rpmssh登陆到客户端，安装Logstash Forwarder
yum localinstall logstash-forwarder-0.4.0-1.x86_64.rpm查看logstash-forwarder的配置文件位置
# rpm -qc logstash-forwarder /etc/logstash-forwarder.conf备份配置文件
cp /etc/logstash-forwarder.conf /etc/logstash-forwarder.conf.save编辑 /etc/logstash-forwarder.conf， servers，你需要根据你的情况进行修改。
cat > /etc/logstash-forwarder.conf << EOF { "network": { "servers": [ "elk.chenshake.com:5000" ], "ssl ca": "/etc/pki/tls/certs/logstash-forwarder.crt", "timeout": 15 }, "files": [ { "paths": [ "/var/log/messages", "/var/log/secure" ], "fields": { "type": "syslog" } } ] } EOF
启动服务
systemctl enable logstash-forwarder systemctl start logstash-forwarder systemctl status logstash-forwarder查看日志，确认服务正常
kibana设置
访问IP，就可以出现Kibana的页面
选择create，点击上面的Discover
你可以尝试ssh退出登录，再ssh进去，马上就可以在日志里看到你ssh登录的事件。
增加节点
增加客户端的安装，只需要把Logstash Forwarder安装和配置都是一样的。你需要额外多做一步的就是
把服务器上的 /etc/pki/tls/certs/logstash-forwarder.crt 复制到目标节点的相同的目录下就可以。
参考资料
https://www.digitalocean.com/community/tutorials/how-to-use-logstash-and-kibana-to-centralize-logs-on-centos-7
https://media-glass.es/2015/04/19/installing-an-elk-stack-on-centos-7/
https://gist.github.com/ashrithr/c5c03950ef631ac63c43