docker-compose搭建fluentd+elasticsearch+kibana日志记录服务

以下文章建立在已经安装完Docker与docker-compose环境的前提下,若还未安装可以参考文章https://www.jianshu.com/p/7fa86b175c20

我们平常查询docker容器运行日志比较麻烦，我们通过docker的fluentd日志驱动将数据发送到fluentd，fluentd将数据转发到elasticsearch，再通过kibana可视化查询容器日志。

以下文件是搭建efk日志系统的docker-compose.yml文件

version: '2'
services:
  fluentd:
    image: registry.cn-hangzhou.aliyuncs.com/lytech/fluentd
    links:
      - "elasticsearch"
    ports:
      - "24224:24224"
      - "24224:24224/udp"
    logging:
        driver: "json-file"
        options:
            max-size: 100m
            max-file: "5"
    restart: always

  elasticsearch:
    image: elasticsearch:6.6.2
    container_name: elasticsearch
    ports:
      - "9200:9200"
    environment:
      - "discovery.type=single-node"
      - "cluster.name=docker-cluster"
      - "bootstrap.memory_lock=true"
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    restart: always
    volumes:
      - ./data:/usr/share/elasticsearch/data

  kibana:
    image: kibana:6.6.2
    container_name: kibana
    links:
      - "elasticsearch"
    ports:
      - "5601:5601"
    restart: always

在docker-compose.yml目录下运行docker-compose up -d即可启动日志系统，稍等一分钟访问http://IP:5601(其中IP替换为运行服务的宿主机IP地址)，可以看到以下界面

image.png

在Index pattern中填入fluentd-*即可查询fluentd转发的日志

image.png

image.png

在任意docker服务的docker-compose.yml 中加入日志存储设置

version: '2'
services:
  nginx:
    image: nginx
    ports:
      - "80:80"
    logging:
      driver: "fluentd"
      options:
        fluentd-address: 192.168.31.117:24224
        tag: nginx
    restart: always

最后可在点击侧边栏的discovery即可看到搜集到的日志信息

image.png