docker部署logstash

#### 前言

logstash的部署方式采用docker

 

#### 准备工作

##### 时区文件

保证容器服务的时间与宿主机的时间一致

```

cat > /etc/timezone <<-EOF
Asia/Shanghai

EOF

```

 

##### logstash.yml文件

```

cat > /data/logstash/config/logstash.yml <<-EOF
http.host: 0.0.0.0

xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.hosts: url/ip:9200   # 建议: 云主机上部署，url采用内网的url，同理，ip也是采用内网的ip
xpack.monitoring.elasticsearch.username: elasticsearch_username
xpack.monitoring.elasticsearch.password: elasticsearch_password

EOF

```

 

##### logstash.conf文件

```

cat > /data/logstash/pipeline/logstash.conf <<-EOF
input {
  beats {
    port => 5044    # 本机部署的logstash端口，注:是容器暴露在宿主机的端口
    codec => plain { charset => "UTF-8" }    #  由于从filebeat段推送到logstash的日志文件不是json格式的，同时存在中文字符，故采用plain格式，并对数据进行UTF-8编码转换
  }
}

filter {
  grok {   
    match => { "message" => "\[(?\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s\+\d{4})\]\s+\[(?\d+)\]\s+\[(?[a-zA-Z]*)\]\s+(?.*)" }
  }
}

output {
  elasticsearch {
    action => "index"
    hosts => ["url/ip:9200"]    # 建议: 云主机上部署，url采用内网的url，同理，ip也是采用内网的ip
    index => "logstash-dev-img-%{+YYYYMMdd}"    # logstash-dev-img，此内容，可自定义
    user => "elasticsearch_username"
    password => "elasticsearch_password"
  }
}

EOF

```

注释：用于json格式化文件的input写法

```

input {
  tcp {
    port => 5044
    codec => "json_lines"    # 每行读取json序列化数据
  }
}

```

 

#### 部署方式1:sh脚本形式部署docker服务

```

cat > docker-logstash-root.sh <<-EOF
#!/usr/bin/env bash

docker run -d \
  --privileged=true \
  -u root \
  --name logstash \
  --restart always \
  -p 5044:5044 \
  -v /etc/localtime:/etc/localtime \
  -v /etc/timezone:/etc/timezone \
  -v /data/logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml \
  -v /data/logstash/pipeline/logstash.conf:/usr/share/logstash/pipeline/logstash.conf \
  docker.elastic.co/logstash/logstash:7.6.2

EOF

```

> bash docker-logstash-root.sh

 

#### 部署方式2:docker-compose形式部署docker服务

```

cat > docker-compose.yml <<-EOF
version: "3.5"
services:
  logstash:
    image: docker.elastic.co/logstash/logstash:7.6.2
    container_name: logstash
    hostname: logstash-
    privileged: true
    user: root
    ports:
      - 5044:5044
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /etc/timezone:/etc/timezone:ro
      - /data/logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml:ro
      - /data/logstash/pipeline/logstash.conf:/usr/share/logstash/pipeline/logstash.conf:ro
    restart: always
    tty: true

EOF

```

> docker-compose up -d