马哥教育N36第二十四周作业

一、简述elasticsearch、logstash、kibana、filebeat的特点，并画图表述在elk里面的作用。

Elastic Stack 各组件作用：

1. Filebeat

它是 Lightweight Shipper for Logs，Filebeat 能够将数据简单的传送到 Logstash 或 ElasticSearch。

• 因为它基本上不对日志数据做任何处理，所以资源占用非常少。
• Filebeat 读取日志数据如果因为某些原因中断，它将记录当前的位置，当重新上线时将从记录的位置开始读取。
• 它还使用一种 backpressure-sensitive 协议传送数据，根据 Logstash 的繁忙程度，调节发送数据的速度。

2. Logstash

logstash 动态地接收、转换和传送您的数据，而不考虑格式或复杂性。

• Ingest Data of All Shapes, Sizes, and Sources
• Parse & Transform Your Data On the Fly
• Choose Your Stash, Transport Your Data
• Create and configure your pipeline, your way
• 支持通过一个 UI 中心化管理和部署

3. ElasticSearch

ElasticSearch 是一个分布式、安静搜索和分析引擎，现在能够处理越来越多的实际用例。作为 Elastic Stack 的核心，它集中存储数据，这样就可以发现预期的数据以及发现隐藏的非预期数据。

• 基于 Lucene 搜索引擎技术开发而来
• 支持和多种编程语言交互，比如 Java, Python, .NET, SQL, and PHP 等。
• 分布式存储数据

4. Kibana

Kibana 可视化 ElasticSearch 数据并在 Elastic Stack 中导航您可以执行任何操作，从跟踪查询负载到了解请求在应用程序中的流动方式。

• A picture’s worth a thousand log lines
• 支持分享图表数据
• Beautifully secure

二、使用elasticsearch、logstash、kibana、filebeat搭建日志收集系统来收集nginx和tomcat日志。

1. 简易实验网络拓扑图

2. 实验前准备

• 准备安装用软件包
  到官方网站下载最新版的rpm包。下载地址：https://www.elastic.co/downloads/

 ~]# ll
-rw-r--r--  1 root root 285118945 Sep  6 08:46 elasticsearch-7.3.1-x86_64.rpm
-rw-r--r--  1 root root  25027402 Sep  6 08:46 filebeat-7.3.1-x86_64.rpm
-rw-r--r--  1 root root 243650898 Sep  6 08:46 kibana-7.3.1-x86_64.rpm
-rw-r--r--  1 root root 172931780 Sep  6 08:46 logstash-7.3.1.rpm

• 系统初始化

# 关闭selinux和防火墙
sed -i '/SELINUX/s/enforcing/disabled/' /etc/selinux/config
systemctl disable firewalld

# 打开fd数量和内存锁限制
echo "* soft nofile 65536" >> /etc/security/limits.conf
echo "* hard nofile 65536" >> /etc/security/limits.conf
echo "* soft memlock unlimited" >> /etc/security/limits.conf
echo "* hard memlock unlimited" >> /etc/security/limits.conf

# 安装常用工具和同步时间
yum install -y net-tools vim lrzsz tree screen lsof tcpdump wget ntpdate
echo "*/5 * * * * ntpdate time1.aliyun.com &> /dev/null && hwclock -w" >> /var/spool/cron/root
timedatectl set-ntp true

# 重启服务器
reboot

• 各ES 服务器准备数据目录

mkdir /elkdata/{data,logs} -pv
chown -R elasticsearch.elasticsearch /elkdata/

3. 安装ElasticSearch

• 配置java环境
  由于最新版本已经自带了java环境，所以不需要额外安装软件包了。

• 安装 es
  基本不用安装其他依赖包

[root@elk1 ~]# yum install ./elasticsearch-7.3.1-x86_64.rpm 
[root@elk2 ~]# yum install ./elasticsearch-7.3.1-x86_64.rpm 
[root@elk3 ~]# yum install ./elasticsearch-7.3.1-x86_64.rpm 

==================================================================================================================================================================================================================
 Package                                           Arch                                       Version                                       Repository                                                       Size
==================================================================================================================================================================================================================
Installing:
 elasticsearch                                     x86_64                                     7.3.1-1                                       /elasticsearch-7.3.1-x86_64                                     458 M

Transaction Summary
==================================================================================================================================================================================================================
Install  1 Package

Total size: 458 M

• java的安装目录验证

~]# cd /usr/share/elasticsearch/jdk/
jdk]# ll
---------------------------------------
total 24
drwxr-xr-x  2 root root 4096 Sep  6 08:47 bin
drwxr-xr-x  5 root root  123 Sep  6 08:47 conf
drwxr-xr-x  3 root root  132 Sep  6 08:47 include
drwxr-xr-x  2 root root 4096 Sep  6 08:47 jmods
drwxr-xr-x 72 root root 4096 Sep  6 08:47 legal
drwxr-xr-x  5 root root 4096 Sep  6 08:47 lib
-rw-r--r--  1 root root 1190 Aug 20 04:20 release
---------------------------------------
jdk]# ./bin/java -version
openjdk version "12.0.2" 2019-07-16
OpenJDK Runtime Environment (build 12.0.2+10)
OpenJDK 64-Bit Server VM (build 12.0.2+10, mixed mode, sharing)

• 修改es配置文件

~]# grep ^[a-zA-Z] /etc/elasticsearch/elasticsearch.yml 
cluster.name: myelk
node.name: node3
path.data: /elkdata/data
path.logs: /elkdata/logs
network.host: 192.168.30.102
http.port: 9200
discovery.seed_hosts: ["192.168.30.105:9300"]
cluster.initial_master_nodes: ["node1", "node2", "node3"]

• 查看jvm设置

grep ^[-] /etc/elasticsearch/jvm.options 
---------------------------------------
-Xms1g
-Xmx1g
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly
-Des.networkaddress.cache.ttl=60
-Des.networkaddress.cache.negative.ttl=10
-XX:+AlwaysPreTouch
-Xss1m
-Djava.awt.headless=true
-Dfile.encoding=UTF-8
-Djna.nosys=true
-XX:-OmitStackTraceInFastThrow
-Dio.netty.noUnsafe=true
-Dio.netty.noKeySetOptimization=true
-Dio.netty.recycler.maxCapacityPerThread=0
-Dlog4j.shutdownHookEnabled=false
-Dlog4j2.disable.jmx=true
-Djava.io.tmpdir=${ES_TMPDIR}
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/var/lib/elasticsearch
-XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log
---------------------------------------

• 启动服务

systemctl daemon-reload
systemctl start elasticsearch
systemctl enable elasticsearch

• 验证es是否启动

~]# curl 192.168.30.105:9200
{
  "name" : "node1",
  "cluster_name" : "myelk",
  "cluster_uuid" : "Kb3xH0fnR5KanAp438a6fQ",
  "version" : {
    "number" : "7.3.1",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "4749ba6",
    "build_date" : "2019-08-19T20:19:25.651794Z",
    "build_snapshot" : false,
    "lucene_version" : "8.1.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

• 可能遇到的问题

ERROR: [1] bootstrap checks failed
elasticsearch: [1]: memory locking requested for elasticsearch process but memory is not locked

①方式一
在配置文件中注释对应的配置可以解决这个问题
#bootstrap.memory_lock: true

②方式二
我开始因为安装了java的openjdk，后来我将虚拟机还原后从新安装，开启锁定内存并没有发生错误，所以将实验环境还原后再试试。而且最新版的启动脚本中内存大小没有限制了，不需要修改。
开启日志的方式，去掉最后的–quiet

vim /usr/lib/systemd/system/elasticsearch.service
ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid  
...
LimitAS=infinity
...

4. 安装 elasticsearch-head

• 克隆源代码
  项目地址：https://github.com/mobz/elasticsearch-head

git clone https://github.com/mobz/elasticsearch-head.git

• 安装

yum install npm -y
cd elasticsearch-head/
npm install grunt -save
npm install

• 启动

npm run start &

• 使用head插件
  配置所有 elasticsearch 允许远程访问，再配置文件末尾添加如下配置

vim /etc/elasticsearch/elasticsearch.yml
http.cors.enabled: true
http.cors.allow-origin: "*"

• 可能遇到的问题
  从节点无法加入主节点，这是因为我把另外两个虚拟机还原了重新安装，这个虚拟机由于启动服务成功九没有还原初始化。
  解决方法是：删除/elkdata/data 目录的所有数据，重启服务九可以了。

elasticsearch: Caused by: java.lang.IllegalStateException: failure when sending a validation request to node

• 测试索引创建
  

5. 配置web服务器

• 下载最新版的源代码并安装Nginx

~]# ll
-rw-r--r--  1 root root 1032630 Sep  5 15:15 nginx-1.16.1.tar.gz
cp nginx-1.16.1.tar.gz /usr/local/src/
cd /usr/local/src/
tar xf nginx-1.16.1.tar.gz
./configure --prefix=/usr/local/nginx
make && make install

• 创建nginx测试页面

cd /usr/local/nginx/
mkdir html/test
vim html/test/index.html
-------------------------------
 Nginx test page! 
-------------------------------

• 编写配置文件

vim conf/web.conf
-------------------------------
server {
	server_name www.ilinux.io;
	listen 80;
	location /test {
		root html;
		index index.html index.htm;
	}
}
-------------------------------

• 测试效果
  

6. 在Nginx主机上安装Logstash

• 配置 java 环境
  Logstash requires Java 8 or Java 11. 一定要安装对应的版本，版本高了会有问题。

yum install jdk-8u221-linux-x64.rpm
~]# vim /etc/profile.d/java.sh
-------------------------------
export JAVA_HOME=/usr/java/default
export CLASSPATH=.:$JAVA_HOME/jre/lib:$JAVA_HOME/lib
export PATH=$PATH:$JAVA_HOME/bin
-------------------------------
. /etc/profile.d/java.sh
~]# java -version
-------------------------------
java version "1.8.0_221"
Java(TM) SE Runtime Environment (build 1.8.0_221-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.221-b11, mixed mode)
-------------------------------

• 安装 Logstash

~]# yum install ./logstash-7.3.1.rpm 
==================================================================================================================================================================================================================
 Package                                          Arch                                           Version                                            Repository                                               Size
==================================================================================================================================================================================================================
Reinstalling:
 logstash                                         noarch                                         1:7.3.1-1                                          /logstash-7.3.1                                         288 M

Transaction Summary
==================================================================================================================================================================================================================
Reinstall  1 Package

• 配置 Logstash 测试

vim /etc/logstash/conf.d/stdout.conf
input {
	stdin {}
}

output {
	stdout {
		codec => rubydebug
	}
}

• 测试效果

/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/stdout.conf 
hello
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/awesome_print-1.7.0/lib/awesome_print/formatters/base_formatter.rb:31: warning: constant ::Fixnum is deprecated
{
       "message" => "hello",
          "host" => "websrv1.ilinux.io",
    "@timestamp" => 2019-09-09T01:24:00.684Z,
      "@version" => "1"
}

• 测试和检查语法

/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/stdout.conf -t

7. 收集Nginx的日志并传送到es中

• 配置Nginx的日志格式

vim /usr/local/nginx/conf/nginx.conf
-------------------------------
  log_format json_log '{"@timestamp":"$time_iso8601",'
    '"host":"$server_addr",'
    '"clientip":"$remote_addr",'
    '"size":$body_bytes_sent,'
    '"responsetime":$request_time,'
    '"upstreamtime":"$upstream_response_time",'
    '"upstreamhost":"$upstream_addr",'
    '"http_host":"$host",'
    '"url":"$uri",'
    '"domain":"$host",'
    '"xff":"$http_x_forwarded_for",'
    '"referer":"$http_referer",'
    '"status":"$status"}';

   access_log  logs/access.log  json_log;
-------------------------------

• 重新启动Nginx

sbin/nginx -t
sbin/nginx -s reload

• 验证日志格式

logs]# cat access.log 
192.168.30.88 - - [09/Sep/2019:08:32:47 +0800] "GET /test HTTP/1.1" 301 169 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36"
192.168.30.88 - - [09/Sep/2019:08:32:47 +0800] "GET /test/ HTTP/1.1" 200 28 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36"
192.168.30.88 - - [09/Sep/2019:08:32:47 +0800] "GET /favicon.ico HTTP/1.1" 404 555 "http://192.168.30.100/test/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36"
{"@timestamp":"2019-09-09T09:37:29+08:00","host":"192.168.30.100","clientip":"192.168.30.88","size":28,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"192.168.30.100","url":"/test/index.html","domain":"192.168.30.100","xff":"-","referer":"-","status":"200"}
{"@timestamp":"2019-09-09T09:37:29+08:00","host":"192.168.30.100","clientip":"192.168.30.88","size":555,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"192.168.30.100","url":"/favicon.ico","domain":"192.168.30.100","xff":"-","referer":"http://192.168.30.100/test/","status":"404"}

• 使用在线工具验证json格式是否正确
  
• 修改 Logstash配置

~]# vim /etc/logstash/conf.d/nginx.conf
input {
	file {
		type => "nginx_access_log"
		path => "/usr/local/nginx/logs/access.log"
		start_position => "beginning"
	}
}

output {
	elasticsearch {
		hosts => ["192.168.30.105:9200"]
		index => "nginx-accesslog-%{+yyyy.MM.dd}"
	}
}

• 检查语法

~]# /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/nginx.conf -t
Thread.exclusive is deprecated, use Thread::Mutex
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[WARN ] 2019-09-09 09:51:38.599 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2019-09-09 09:51:40.058 [LogStash::Runner] Reflections - Reflections took 45 ms to scan 1 urls, producing 19 keys and 39 values 
Configuration OK
[INFO ] 2019-09-09 09:51:40.528 [LogStash::Runner] runner - Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash

• 在es中查看索引

/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/nginx.conf

8. 安装和配置Filebeat

• 安装
  在web 端时时收集日志并传递给logstash 进一步处理

yum install ./filebeat-7.3.1-x86_64.rpm

• 配置 Filebeat
  5044端口是logstash开启的，所以要正确配置logstash

vim /etc/logstash/logstash.yml
--------------------------------------------------
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /usr/local/nginx/logs/access.log
  document_type: nginx-access-log-001
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 1
setup.kibana:
output.logstash:
  hosts: ["192.168.30.100:5044"]
processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
 --------------------------------------------------

• 配置logstash

conf.d]# cat nginx.conf 
input {
	beats {
		port => 5044
		codec => "json"
	}
}

output {
	elasticsearch {
		hosts => ["192.168.30.105:9200"]
		index => "nginx-accesslog-%{+yyyy.MM.dd}"
	}
}

• 清空测试数据重新读取日志
  

9. 安装 Kibana

• 安装
  在三个es主机中选择一台来安装Kibana

yum install ./kibana-7.3.1-x86_64

• 配置

~]# grep ^[^#] /etc/kibana/kibana.yml 
server.host: "192.168.30.106"
elasticsearch.hosts: ["http://192.168.30.105:9200"]

• 启动并验证结果
  Kibana 默认是监听在5601端口上

systemctl restart kibana

三、分别使用lxc容器和docker容器搭建nginx服务，能够正常访问到容器内的web服务

1. 使用lxc容器

• 安装

yum -y install lxc lxc-templates lxc-doc libvirt

• 检查配置

lxc-checkconfig

• 创建容器
  root的初始密码在 /var/lib/lxc/nginx/tmp_root_pass

lxc-create -n nginx -t centos

• 运行容器
  进入容器修改密码，并配置yum源，安装nginx并启动

lxc-start -n nginx

• 查看状态

lxc-info -n nginx
~]# lxc-info -n nginx
----------------------------------------
Name:           nginx
State:          RUNNING
PID:            44221
IP:             192.168.122.221
CPU use:        0.17 seconds
BlkIO use:      0 bytes
Memory use:     1.11 MiB
KMem use:       0 bytes
Link:           vethHMTE3O
 TX bytes:      22.91 KiB
 RX bytes:      47.21 KiB
 Total bytes:   70.12 KiB
 ----------------------------------------

• 查看 iptables 规则

~]# iptables -vnL
 ----------------------------------------
Chain INPUT (policy ACCEPT 58203 packets, 104M bytes)
 pkts bytes target     prot opt in     out     source               destination         
  126  7877 ACCEPT     udp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0            udp dpt:53
    0     0 ACCEPT     tcp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:53
    5  1640 ACCEPT     udp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0            udp dpt:67
    0     0 ACCEPT     tcp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:67

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
19759   44M ACCEPT     all  --  *      virbr0  0.0.0.0/0            192.168.122.0/24     ctstate RELATED,ESTABLISHED
14357  590K ACCEPT     all  --  virbr0 *       192.168.122.0/24     0.0.0.0/0           
    0     0 ACCEPT     all  --  virbr0 virbr0  0.0.0.0/0            0.0.0.0/0           
    0     0 REJECT     all  --  *      virbr0  0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable
    0     0 REJECT     all  --  virbr0 *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable

Chain OUTPUT (policy ACCEPT 39849 packets, 3592K bytes)
 pkts bytes target     prot opt in     out     source               destination         
    5  1640 ACCEPT     udp  --  *      virbr0  0.0.0.0/0            0.0.0.0/0            udp dpt:68
 ----------------------------------------

• 测试nginx
  现在本机访问是可以的，外部主机不能访问。

curl 192.168.122.221

2. 使用Docker搭建Nginx服务

• 配置 yum 源

wget -O /etc/yum.repos.d/docker-ce.repo https://download.docker.com/linux/centos/docker-ce.repo
sed -i 's+download.docker.com+mirrors.tuna.tsinghua.edu.cn/docker-ce+' /etc/yum.repos.d/docker-ce.repo

• 查看版本信息

~]# yum info docker-ce
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
Available Packages
Name        : docker-ce
Arch        : x86_64
Epoch       : 3
Version     : 19.03.2
Release     : 3.el7
Size        : 24 M
Repo        : docker-ce-stable/x86_64
Summary     : The open-source application container engine
URL         : https://www.docker.com
License     : ASL 2.0
Description : Docker is a product for you to build, ship and run any application as a
            : lightweight container.
            : 
            : Docker containers are both hardware-agnostic and platform-agnostic. This means
            : they can run anywhere, from your laptop to the largest cloud compute instance and
            : everything in between - and they don't require you to use a particular
            : language, framework or packaging system. That makes them great building blocks
            : for deploying and scaling web apps, databases, and backend services without
            : depending on a particular stack or provider.

• 安装
  首先要安装 container-selinux ，到 这个网站 下载对应的rpm 安装。

yum install ./container-selinux-2.74-1.el7.noarch.rpm
yum install docker-ce

• 查看 包文件构成

~]# rpm -ql docker-ce
/usr/bin/docker-init
/usr/bin/docker-proxy
/usr/bin/dockerd
/usr/lib/systemd/system/docker.service
/usr/lib/systemd/system/docker.socket
/var/lib/docker-engine/distribution_based_engine-ce.json

• 配置 Docker 镜像加速
  默认这个文件和目录是不存在的。

vim /etc/docker/daemon.json
{
    "registry-mirrors": ["https://registry.docker-cn.com"]
}

• 拉取 Nginx 的docker镜像

 ~]# docker image pull nginx
----------------------------------------------------
Using default tag: latest
latest: Pulling from library/nginx
1ab2bdfe9778: Pull complete 
a17e64cfe253: Pull complete 
e1288088c7a8: Pull complete 
Digest: sha256:53ddb41e46de3d63376579acf46f9a41a8d7de33645db47a486de9769201fec9
Status: Downloaded newer image for nginx:latest
docker.io/library/nginx:latest
----------------------------------------------------

• 运行镜像
  后台运行nginx，并绑定端口

~]# docker container run -d --name web -p 192.168.30.108:80:80 nginx
9f37e811a8c564c6c66bdb783d8bcbf452da12de24cff641dafc58787d7306ea

 ~]# ss -tnlp
State      Recv-Q Send-Q                                                            Local Address:Port                                                                           Peer Address:Port              
LISTEN     0      128                                                                           *:111                                                                                       *:*                   users:(("rpcbind",pid=6131,fd=4),("systemd",pid=1,fd=59))
LISTEN     0      128                                                              192.168.30.108:80                                                                                        *:*                   users:(("docker-proxy",pid=48881,fd=4))

• 查看端口映射

~]# docker container ls -a
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                     PORTS                       NAMES
9f37e811a8c5        nginx               "nginx -g 'daemon of…"   6 seconds ago       Up 5 seconds               192.168.30.108:80->80/tcp   web
e67203e7c7a5        archlinux/base      "/usr/bin/bash"          10 minutes ago      Exited (0) 8 minutes ago                               archlinux

• 访问nginx
  

四、简述docker网络模型，并实现桥接模式下不同宿主机之间的docker网络互通

1. docker网络模型

• Docker 自动创建的三种网络

~]# docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
db9d7b000dec        bridge              bridge              local
4670816f74ca        host                host                local
4cf33d560c05        none                null                local

• bridge 网络
  ①默认是使用docker0 这个网桥
  ②容器的IP 和 docker0 在同一个网段，并且网关指向 docker0
  ③docker0 是 NAT 桥
  ④外界访问容器内的服务，需要通过宿主机的套接字访问

• host 网络
  ①容器的IP使用的是宿主机的IP
  ②容器服务的端口不能和宿主机的端口冲突

• none 网络
  ①容器只有 lo 网络接口
  ②容器不与外界通信
  ③常用来分析和处理数据

• 容器共享网络
  ①这种网络创建的容器共享已经存在的容器的网络
  ②两个容器可以通过 lo 设备通信
  ③除网络以外，其他资源是相互隔离的

2. docker网络互通

• 宿主机打开转发

~]# vim /etc/sysctl.conf 
net.ipv4.ip_forward = 1

~]# sysctl -p

• 为各自的容器绑定端口
  语法如下：

docker run -p :: image_name

• 映射随机端口

docker run -p 80 image_name

• 查看端口映射

docker port container_name

• 使用各自宿主机的套接字访问对方的容器

参考文章

• ElasticSearch 5 安装部署常见错误或问题

• Install Elasticsearch with RPM

• ELK-7.3安装部署

• Elasticse报错-java.lang.IllegalStateException: failure when sending a validation request to node

• Important discovery and cluster formation settings

• Kibana server is not ready yet出现的原因