使用docker-compose与SpringBoot搭建ELK日志分析系统

使用docker-compose与SpringBoot搭建ELK日志分析系统

ELK对应Elasticsearch、Logstash、Kibana
Logstash作为日志采集工具，向Elasticsearch写日志信息；
Elasticsearch提供存储与检索功能；
Kibana为Elasticsearch的查询接口，提供友好的图形界面。

搭建ELK环境

这里使用docker-compose把ELK作为一组项目容器启动，这里请提前搭好docker、docker-compose环境。

新建 /data/elk 目录
在elk目录下创建文件 docker-compose配置文件
docker-compose.yml

version: '2'
services:
  elasticsearch:
    image: elasticsearch
    environment:
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    volumes:
      - $PWD/elasticsearch/data:/usr/share/elasticsearch/data
    hostname: elasticsearch
    restart: always
    ports:
      - "9200:9200"
      - "9300:9300"
  kibana:
    image: kibana
    environment:
      - ELASTICSEARCH_URL=http://elasticsearch:9200 #elasticsearch查询接口地址
    hostname: kibana
    depends_on:
      - elasticsearch  #后于elasticsearch启动
    restart: always
    ports:
      - "5601:5601"
  logstash:
    image: logstash
    command: logstash -f /etc/logstash/conf.d/logstash.conf  #logstash 启动时使用的配置文件
    volumes:
      - $PWD/logstash/conf.d:/etc/logstash/conf.d  #logstash 配文件位置
      - $PWD/logst:/tmp
    hostname: logstash
    restart: always
    depends_on:
      - elasticsearch  #后于elasticsearch启动
    ports:
      - "7001-7005:7001-7005"
      - "4560:4560"
      - "9600:9600"

创建logstash启动配置文件
/data/elk/logstash/conf.d/logstash.conf

input {
    tcp {
        mode => "server"
        host => "0.0.0.0"   //日志输入地址(所有外网地址)，也指定具体输入地址
        port => 4560      //日志输入端口
        codec => json_lines
    }
}
output{
  elasticsearch {
    hosts => ["elasticsearch:9200"]    
    action => "index"
    index => "applog"
    }
  stdout {
    codec => rubydebug
    }
}

如果elk 3个镜像下不来，可以配置国内的镜像加速，如阿里的、docker中国官方的
/etc/docker/daemon.json
{
“registry-mirrors”: [“https://registry.docker-cn.com”]
}

启动、停止容器组

#/data/elk 目录下
## 启动
[root@localhost elk]# docker-compose up -d 
Creating network "elk_default" with the default driver
Creating elk_elasticsearch_1_62f42e83ab51 ... done
Creating elk_logstash_1_b4f112872efa      ... done
Creating elk_kibana_1_dd80d748dac1        ... done
[root@localhost elk]# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                                                              NAMES
e8715946efd3        kibana              "/docker-entrypoin..."   3 seconds ago       Up 2 seconds        0.0.0.0:5601->5601/tcp                                                             elk_kibana_1_c07ef0b34a97
0cfc13d30a68        logstash            "/docker-entrypoin..."   3 seconds ago       Up 2 seconds        0.0.0.0:4560->4560/tcp, 0.0.0.0:7001-7005->7001-7005/tcp, 0.0.0.0:9600->9600/tcp   elk_logstash_1_6abacb9c09ba
7bf21946fb20        elasticsearch       "/docker-entrypoin..."   4 seconds ago       Up 3 seconds        0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp                                     elk_elasticsearch_1_47fc541fb48d


## 停止容器组
[root@localhost elk]# docker-compose down
Stopping elk_kibana_1_c07ef0b34a97        ... done
Stopping elk_logstash_1_6abacb9c09ba      ... done
Stopping elk_elasticsearch_1_47fc541fb48d ... done
Removing elk_kibana_1_c07ef0b34a97        ... done
Removing elk_logstash_1_6abacb9c09ba      ... done
Removing elk_elasticsearch_1_47fc541fb48d ... done
Removing network elk_default
[root@localhost elk]# 

配置SpringBoot应用向Logstash输入日志

（1）添加 logstash-logback 依赖包

<dependency>
      <groupId>net.logstash.logbackgroupId>
      <artifactId>logstash-logback-encoderartifactId>
      <version>4.9version>
dependency>

（2）在resource目录添加logback配置
logback.xml

<configuration>
    <include resource="org/springframework/boot/logging/logback/base.xml" />

    <appender name="LOGSTASH" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
        
        
        <destination>192.168.10.128:4560destination>
        <encoder charset="UTF-8" class="net.logstash.logback.encoder.LogstashEncoder" />
    appender>

    <root level="INFO">
        <appender-ref ref="LOGSTASH" />
        <appender-ref ref="CONSOLE" />
    root>

configuration>

Kibana请求Elasticsearch日志结果

启动SpringBoot应用，即可以在Kibana查到对应的信息