本文主要记录下以前搭建ELK的docker镜像过程,记录下docker基础使用,如何基于centos的基础镜像构建应用。
准备jdk 放在tools文件夹下
准备Dockerfile
FROM centos:latest
MAINTAINER Yampery
VOLUME [ "/opt/product/data/" ]
RUN /bin/cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
RUN /bin/echo -e "ZONE="Asia/Shanghai"\nUTC=false\nRTC=false" > /etc/sysconfig/clock
RUN mkdir /opt/product/tools/
ADD ./tools /opt/product/tools/
ENV JAVA_HOME /opt/product/tools/jdk-9.0.1
CMD ["/usr/sbin/init"]
构建文件结构
|-elk
|-tools
|-jdk-9.0.1
|--Dockerfile
构建镜像
elk/elkbase$ docker build -t elkbase:v1.0 .
准备elasticsearch
https://www.elastic.co/downloads/elasticsearch
准备Dockerfile
FROM elkbase:v1.0
MAINTAINER Yampery
VOLUME [ "/opt/product/data/" ]
ADD ./tools /opt/product/
RUN useradd elk && chown -R elk:elk /opt/product/elasticsearch-6.0.0
ADD build.sh /root/
RUN chmod +x /root/build.sh
EXPOSE 9200
EXPOSE 9300
ENTRYPOINT ["/root/build.sh"]
CMD ["/usr/sbin/init"]
准备脚本build.sh
#!/bin/bash
echo "* soft nofile 65536" > /etc/security/limits.conf
echo "* hard nofile 131072" > /etc/security/limits.conf
echo "* soft nproc 2048" > /etc/security/limits.conf
echo "* hard nproc 4096" > /etc/security/limits.conf
echo "vm.max_map_count=655360" > /etc/sysctl.conf
sysctl -p
cd /opt/product/elasticsearch-6.0.0/config/
rm -rf elasticsearch.yml
cp /opt/product/data/elk/elasticsearch.yml .
chown -R elk:elk /opt/product/elasticsearch-6.0.0
chown -R elk:elk /opt/product/data/elk/elasticsearchdata
su - elk <
-
构建目录结构
|-elk
|-elasticsearch
|-tools
|--Dockerfile
|--build.sh
-
构建镜像
elk/elasticsearch$ docker build -f elasticsearch:v1.0 .
-
配置
在/opt/product/data目录下建立elk目录,并拷贝elasticsearch.yml文件到该目录
path.data: /opt/product/data/elk/elasticsearchdata
network.host: 0.0.0.0
在/opt/product/data/elk目录 创建 elasticsearchdata 目录
-
运行容器
docker run --name elasticsearch --privileged --restart=always -d -ti -v /opt/product/data:/opt/product/data -p 9200:9200 -p 9300:9300 elasticsearch:v1.0 /bin/bash
-> 访问:ip:9200/
logstash
-
准备logstash
https://www.elastic.co/downloads/logstash
-
准备Dockerfile
FROM elkbase:v1.0
MAINTAINER Yampery
VOLUME [ "/opt/product/data/" ]
ADD ./tools /opt/product/
ADD build.sh /root/
RUN chmod +x /root/build.sh
EXPOSE 5044
EXPOSE 4560
EXPOSE 8080
ENTRYPOINT ["/root/build.sh"]
CMD ["/usr/sbin/init"]
-
准备脚本 build.sh
#!/bin/bash
export JAVA_HOME=/opt/product/tools/jdk-9.0.1
export PATH=$JAVA_HOME/bin:$PATH
JAVA_OPTS="$JAVA_OPTS -Dfile.encoding=UTF8 -Duser.timezone=GMT+08"
cd /opt/product/logstash-6.0.0/config/
rm -rf logstash.yml
cp /opt/product/data/elk/logstash.yml logstash.yml
/opt/product/logstash-6.0.0/bin/logstash -f /opt/product/data/elk/logstash.conf
-
目录结构
|-elk
|-logstash
|-tools
|--Dockerfile
|--build.sh
-
构建镜像
elk/logstash$ docker build -f logstash:v1.0 .
-
配置logstash
// 在/opt/product/data/elk目录下创建logstash.conf
input {
beats {
port => "5044"
}
}
output {
elasticsearch {
hosts => ["elasticsearch的ip:9200"]
index => "logstash-tomcat-accesslog-%{+YYYY.MM.dd}"
}
}
// 将logstash本身的logstash.yml 拷贝到/opt/product/data/elk目录下
// 在/opt/product/data/elk目录下建立 logstashdata目录
-
启动
docker run --name logstash --restart=always -d -ti -v /opt/product/data:/opt/product/data -p 5044:5044 -p 4560:4560 -p 9090:9090 logstash:v1.0 /bin/bash
kibana
-
准备kibana
https://artifacts.elastic.co/downloads/kibana/kibana-6.0.0-linux-x86_64.tar.gz
-
准备Dockerfile
FROM elkbase:v1.0
MAINTAINER Yampery
VOLUME [ "/opt/product/data/" ]
ADD ./tools /opt/product/
ADD build.sh /root/
RUN chmod +x /root/build.sh
EXPOSE 5601
ENTRYPOINT ["/root/build.sh"]
CMD ["/usr/sbin/init"]
-
准备脚本 build.sh
#!/bin/bash
export JAVA_HOME=/opt/product/tools/jdk-9.0.1
export PATH=$JAVA_HOME/bin:$PATH
cd /opt/product/kibana-6.0.0/config/
rm -rf kibana.yml
ln -s /opt/product/data/elk/kibana.yml .
cd /opt/product/kibana-6.0.0/
rm -rf data
ln -s /opt/product/data/elk/kibanadata /opt/product/kibana-6.0.0/data
/opt/product/kibana-6.0.0/bin/kibana
-
构建镜像
elk/kibana$ docker build -t kibana:v1.0 .
-
配置kibana
// 复制kibana.yml到/opt/product/data/elk 下
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.url: "http://ip:9200"
-
启动
docker run --name kibana -d -ti -v /opt/product/data:/opt/product/data -p 5601:5601 kibana:v1.0 /bin/bash
-
访问
ip:5601/
FileBeat
-
准备FileBeat
https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.0.0-linux-x86_64.tar.gz
-
准备Dockerfile
FROM elkbase:v1.0
MAINTAINER Yampery
VOLUME [ "/opt/product/data/" ]
ADD ./tools /opt/product/
ADD build.sh /root/
RUN chmod +x /root/build.sh
ENTRYPOINT ["/root/build.sh"]
CMD ["/usr/sbin/init"]
-
准备脚本 build.sh
#!/bin/bash
export JAVA_HOME=/opt/product/tools/jdk-9.0.1
export PATH=$JAVA_HOME/bin:$PATH
cd /opt/product/filebeat-6.0.0/
rm -rf filebeat.yml
ln -s /opt/product/data/elk/filebeat.yml .
rm -rf data
ln -s /opt/product/data/elk/filebeatdata /opt/product/filebeat-6.0.0/data
/opt/product/filebeat-6.0.0/filebeat -e -c filebeat.yml
-
构建镜像
elk/filebeat$ docker build -t filebeat:v1.0 .
-
配置filebeat
// 在/opt/product/data/elk目录下 创建filebeat.yml
filebeat.prospectors:
- input_type: log
document_type: tomcataccess
paths:
- /opt/product/data/logs/tomcat/localhost_access_log*.txt
- /opt/product/data/epg2logs/tomcat/localhost_access_log*.txt
output.logstash:
// The Logstash hosts
hosts: ["ip:5044"]
// 在/opt/product/data/elk 目录下创建filebeatdata
-
启动
docker run --name filebeat -d -ti -v /opt/product/data:/opt/product/data filebeat:v1.0 /bin/bash
x-pack
-
安装
./elasticsearch-plugin install x-pack
-
设置密码
https://www.elastic.co/guide/en/x-pack/current/setting-up-authentication.html#set-built-in-user-passwords
./setup-passwords interactive
Enter password for [elastic]:
Reenter password for [elastic]:
Enter password for [kibana]:
Reenter password for [kibana]:
Enter password for [logstash_system]:
Reenter password for [logstash_system]:
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [elastic]
-
logstash配置
input { stdin { } }
output {
elasticsearch {
hosts => ["192.168.107.23:9200"]
user => elastic
password => elastic
}
stdout { codec => rubydebug }
}
同理,kibana也可以配置,调用es接口也需要用户密码
说明
脚本和启动项参数中的/opt/product/data均以自己要设定的挂载目录一致
你可能感兴趣的:(docker,docker,elk,elasticsearch,Logstash,Filebeat)