简单搭建ELK

1、首先拉软件包,基于java语言开发首先配置java环境,centos7不用、推荐4G运行内存

2、首先安装elasticsearch-6.6.0.rpm

rpm -ivh elasticsearch-6.6.0.rpm 

3、修改elasticsearch配置,并启动elasticsearch

vim /etc/elasticsearch/elasticsearch.yml

systemctl start elasticsearch

4、安装kibana

rpm -ivh kibana-6.6.0-x86_64.rpm

5、修改kibana配置、并启动kibana

vim /etc/kibana/kibana.yml

systemctl start kibana

6、安装logstash过滤收集工具，和客户端有点相似

rpm -ivh logstash-6.6.0.rpm

7、配置logstash、并启动logstash、记得监控日志时给日志加权限

chmod 644 /var/log/messages

vim /etc/logstash/conf.d/system.conf

systemctl start logstash

8、效果出现索引

二、原有基础上
1、监听nginx日志，配置nginx源、yum安装nginx,并启动nginx

vim /etc/yum.repos.d/nginx.repo

[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=0
enabled=1

yum -y install nginx

systemctl start nginx

2、配置logstash配置、定义配置规则NIGNXACCESS，用来配置nginx过滤访问日志，记住加权限

cd /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-patterns-core-4.1.2/patterns/

vim nginx_access

URIPARAM1 [A-Za-z0-9$.+!*'|(){
     },~@#%&/=:;_?\-\[\]]*
NGINXACCESS %{
     IPORHOST:client_ip} (%{
     USER:ident}|- ) (%{
     USER:auth}|-) \[%{
     HTTPDATE:timestamp}\] "(?:%{WORD:verb} (%{NOTSPACE:request}|-)(?: HTTP/%{NUMBER:http_version})?|-)" %{
     NUMBER:status} (?:%{
     NUMBER:bytes}|-) "(?:%{URI:referrer}|-)" "%{GREEDYDATA:agent}"

chmod 644 /var/log/nginx/access.log

vim /etc/logstash/conf.d/system.conf

input{
     
   file{
     
        path => "/var/log/messages"
        type => "system_log"
        start_position => "beginning"
   }
   file{
     
        path => "/var/log/nginx/access.log"
        type => "nginx_log"
        start_position => "beginning"
   }

}

filter{
     
   if [type] == "nginx_log"{
     
        grok {
     
             match => {
      "message" => "%{NGINXACCESS}" }
        }
   }
}

output{
     
   if [type] == "nginx_log"{
     
        elasticsearch{
     
             hosts => ["10.0.0.41:9200"]
             index => "nginx_log_%{+YYYY.MM.dd}"
        }
   } else {
     
        elasticsearch{
     
             hosts => ["10.0.0.41:9200"]
             index => "system_log_%{+YYYY.MM.dd}"
        }
   }
}

3、重启logstash

systemctl restart logstash

4、重启logstash后访问nginx页面产生日志，nginx索引才会出现


4、要记住nginx日志格式是否开启，yum安装日志默认打开

vim /etc/nginx/nginx.conf

nginx页面最好做完之后，刷新几次产生日志