logback+redis+logstash+elasearch+kibana集成

版本说明

1. 系统版本：centos7.3
2. elk版本:6.1.3
3. 主机IP：192.168.111.139
4. redis版本：2.8.2

demo地址

https://github.com/system-integration/elk

es安装

开放以下端口

firewall-cmd --zone=public --add-port=9200/tcp --permanent && firewall-cmd --reload
firewall-cmd --zone=public --add-port=6379/tcp --permanent && firewall-cmd --reload
firewall-cmd --zone=public --add-port=5601/tcp --permanent && firewall-cmd --reload

创建elk普通用户

#root执行
useradd elk -d /elk

修改系统配置

#root执行

cat >> /etc/sysctl.conf <> /etc/security/limits.conf <

elk用户安装jdk

export JAVA_HOME=~/java
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

创建目录

#启动脚本目录
mkdir -p /elk/bin

#日志目录
mkdir -p /elk/log

#数据目录
mkdir -p /elk/data

image

解压安装es修改配置如下

# 配置文件路径 /elk/elasticsearch-6.1.3/config/elasticsearch.yml

network.host: 192.168.111.139
path.data: /elk/data
path.logs: /elk/log

制作启动脚本

cat >> /elk/bin/es-start.sh < /dev/null &
EOF

chmod +x /elk/bin/es-start.sh

启动，查看启动日志

/elk/bin/es-start.sh && tail -200f /elk/log/elasticsearch.log

查看服务

http://192.168.111.139:9200/

image

redis安装

启动redis 6379端口

logstash安装

制作脚本

cat >> /elk/logstash-6.1.3/config/server.conf < "192.168.111.139"
        port => 6379
        data_type => "list"
        key => "logstash"
        threads => 5
        codec => "json"
    }
}
filter {
        json {
            source => "message"
        }
}
output {
    #输出到本地文件
    file {
        path => "/elk/%{type}.log"
        codec => line
        {
        format => "[%{@timestamp}]--[%{level}]--[%{host}]--%{message}"
        }
    }
    elasticsearch {
        action => "index"
        hosts  => ["192.168.111.139:9200"]
        index => "%{type}-%{+YYYY.MM.dd}"
        }

}

EOF

cat >> /elk/bin/logstash-start.sh < /elk/log/logstash.log &
EOF 

chmod +x /elk/bin/logstash-start.sh

启动，查看启动日志

/elk/bin/logstash-start.sh && tail -200f /elk/log/logstash.log

Kibana安装

修改kibana配置

vim /elk/kibana-6.1.3-linux-x86_64/config/kibana.yml

server.host: "192.168.111.139"
elasticsearch.url: "http://192.168.111.139:9200"

制作脚本

cat >> /elk/bin/kibana-start.sh < /elk/log/kibana.log &
EOF

chmod +x /elk/bin/kibana-start.sh

启动，查看启动日志

/elk/bin/kibana-start.sh && tail -200f /elk/log/kibana.log

访问配置

http://192.168.111.139:5601

新建spring boot项目

pom.xml

    4.0.0

    com.example
    demo
    0.0.1-SNAPSHOT
    jar

    elk-demo
    Demo project for Spring Boot

    
        org.springframework.boot
        spring-boot-starter-parent
        2.0.0.RELEASE
         
    

    
        UTF-8
        UTF-8
        1.8
    

    
        
            org.springframework.boot
            spring-boot-starter-web
        

        
            org.springframework.boot
            spring-boot-starter-test
            test
        
        
            org.springframework.boot
            spring-boot-configuration-processor
        
        
            redis.clients
            jedis
        
        
      ch.qos.logback
      logback-classic
    
    

    
        
            
                org.springframework.boot
                spring-boot-maven-plugin
            
        
    

logback.xml

    
        192.168.111.44
        6378
        test2
        test
    
    
        
        
            %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n
        
    
    
        
        0
        
        5120
        
    
    
        
        
    
  

application.properties

logging.config=classpath:logback.xml
#redis host
elk.redis.host=192.168.111.139
#redis port
elk.redis.port=6379
#系统唯一标识（区分不同系统）
elk.service.type=app1
#日志级别
elk.log.level=info

测试

启动spring boot程序

查看日志文件

cd /elk && tail -200f /elk/app1.log

image

配置kibana

![image](http://upload-images.jianshu.io/upload_images/1967881-1062115a1fff3ee4.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)