Elastic Certified Engineer复习记录-集群搭建docker篇
文章目录
- 环境准备
- 下载相关的docker镜像
- 创建一个专用的网络
- 开始编写docker-compose 文件
- 尝试打开xpack相关安全性配置
- 重点总结
- 参考文献：
之前提到集群搭建的时候可以通过多种方式，考试的时候多半使用的是rpm或者tar包的方式，但是在没有足够多资源的时候（囊中羞涩又想白嫖），用docker（docker-compose/k8s）的方式搭建对我们来说是最友好的。
环境准备
安装包括docker、docker-compose以及相关的软件。
brew update
brew install docker
brew install docker-compose
下载相关的docker镜像
docker pull docker.elastic.co/elasticsearch/elasticsearch:7.6.0
docker pull docker.elastic.co/kibana/kibana:7.6.0
这里没有用oss版（纯开源版）主要是为了后面可能会有xpack相关的配置的练习，如果用oss版的话可能xpack的组件整个都没了。
这里注意kibana的版本要和es一致，至少大版本是一致的（7.6.x配7.6.x）否则轻则启动之后不停的报错，重则直接因为版本不兼容而启动失败。
创建一个专用的网络
以避免和本机其他docker组件的网络有冲突。
docker network create bigdata
开始编写docker-compose 文件
既然考试的初衷是模拟真实的使用场景，我们可以考虑先搭建一个多节点（此处是3节点）的es集群。（完整的docker-compose文件后面会添上）
配置组合集群，主要有几个关键配置
1. cluster.name=docker-cluster 声明集群名称
2. discovery.seed_hosts=node2,node3 集群初始化的时候需要彼此保活的节点
3. cluster.initial_master_nodes=node1初始化时候的master节点
4. "ES_JAVA_OPTS=-Xms2g -Xmx2g" es启动的最大/小内存，官方默认配置是512m
5. 内存限制
6. （可选）bootstrap.memory_lock=true 交换区锁定
7. （可选）esdata01:/usr/share/elasticsearch/data文件存储挂载
其他的包括nodename、开放端口之类的都是docker的常规操作，不在这里详细介绍。
最简版docker-compose.yml
version: '3.6'
networks:
bigdata:
external: true // 专用网络
volumes:
esdata01: // 磁盘挂载，主要是data文件
driver: local
esdata02:
driver: local
esdata03:
driver: local
services:
node1: // 单一节点配置
image: docker.elastic.co/elasticsearch/elasticsearch:7.6.0 // 镜像
container_name: node1 // 节点名称，其实可以不写
environment: // 上面提到的那些环境参数
- node.name=node1
- cluster.name=docker-cluster
- discovery.seed_hosts=node2,node3
- cluster.initial_master_nodes=node1
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms2g -Xmx2g"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- esdata01:/usr/share/elasticsearch/data
ports: // 开放端口
- 9200:9200
- 9300:9300
networks: // 使用专用网络组
- bigdata
node2:
image: docker.elastic.co/elasticsearch/elasticsearch:7.6.0
container_name: node2
environment:
- node.name=node2
- cluster.name=docker-cluster
- discovery.seed_hosts=node1,node3
- cluster.initial_master_nodes=node1
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms2g -Xmx2g"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- esdata02:/usr/share/elasticsearch/data
networks:
- bigdata
node3:
image: docker.elastic.co/elasticsearch/elasticsearch:7.6.0
container_name: node3
environment:
- node.name=node3
- cluster.name=docker-cluster
- discovery.seed_hosts=node1,node2
- cluster.initial_master_nodes=node1
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms2g -Xmx2g"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- esdata03:/usr/share/elasticsearch/data
networks:
- bigdata
kibana:
image: docker.elastic.co/kibana/kibana:7.6.0
container_name: kibana
environment:
ELASTICSEARCH_HOSTS: http://node1:9200
depends_on:
- node1
- node2
- node3
external_links:
- node1
- node2
- node3
networks:
- bigdata
ports:
- 5601:5601
可以通过命令docker-compose -f "${filepath}/docker-compose.yml" up -d --build来编译启动
➜ docker docker-compose -f "docker-elasticsearch/docker-compose.yml" up -d --build
Creating node1 ... done
Creating node3 ... done
Creating node2 ... done
Creating kibana ... done
然后通过命令docker ps -as 查看启动状态
➜ docker ps -as
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES SIZE
cc9527af7fee docker.elastic.co/kibana/kibana:7.6.0 "/usr/local/bin/dumb…" 5 hours ago Up 5 hours 0.0.0.0:5601->5601/tcp kibana 135MB (virtual 1.14GB)
0641c9015768 docker.elastic.co/elasticsearch/elasticsearch:7.6.0 "/usr/local/bin/dock…" 16 hours ago Up 16 hours 9200/tcp, 9300/tcp node3 3.47MB (virtual 794MB)
0b1f7b4ae9c1 docker.elastic.co/elasticsearch/elasticsearch:7.6.0 "/usr/local/bin/dock…" 16 hours ago Up 16 hours 0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp node1 5.92MB (virtual 796MB)
a165b7826a35 docker.elastic.co/elasticsearch/elasticsearch:7.6.0 "/usr/local/bin/dock…" 16 hours ago Up 16 hours 9200/tcp, 9300/tcp node2 4.59MB (virtual 795MB)
尝试打开xpack相关安全性配置
这里有个可能会让人很困惑的地方，如果我们在docker-compose文件里只开启xpack的安全性认证xpack.security.enabled: true，启动的时候集群会报错，提示需要把xpack.security.transport.ssl.enabled一并开启，否则就需要关掉xpack的认证。如果我们把ssl的配置开启了之后，集群启动的时候又会报认证失败的错。
那就让我们一步步解决这些东西。
- 关闭那俩配置，让集群正常启动
……省略一些
services:
node1:
environment:
- xpack.security.enabled: false
- xpack.security.transport.ssl.enabled: false
……其他的省略
- 登入其中一个节点，创建认证中心（certificate authority）
进入docker节点docker exec -it node1 bash
[root@1b2b7e0c0734 elasticsearch]# pwd
/usr/share/elasticsearch
用es工具生成证书 ./bin/elasticsearch-certutil ca
[root@1b2b7e0c0734 elasticsearch]# ./bin/elasticsearch-certutil ca
WARNING: An illegal reflective access operation has occurred
……中间一大堆……
Please enter the desired output file [elastic-stack-ca.p12]: // 这一行是指定认证机构文件生成路径，不填默认当前路径
Enter password for elastic-stack-ca.p12 : // 这一行是指定ca密码，不填为空
检查一下结果
[root@1b2b7e0c0734 elasticsearch]# ls -ltr
total 572
-rw-r--r-- 1 elasticsearch root 8164 Feb 6 00:07 README.asciidoc
……一些ES自己的文件……
drwxrwxr-x 1 elasticsearch root 4096 Mar 24 16:34 config
-rw------- 1 root root 2527 Mar 28 15:45 elastic-stack-ca.p12 <----要的就是它
- 创建认证文件
bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12（这文件不在当前路径的话要指定它的路径）
[root@1b2b7e0c0734 elasticsearch]# ./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
WARNING: An illegal reflective access operation has occurred
……中间一大堆…… // 指定证书密码生成的文件密码等等，最终会被生成到下面那个路径
Enter password for CA (elastic-stack-ca.p12) :
Please enter the desired output file [elastic-certificates.p12]:
Enter password for elastic-certificates.p12 :
Certificates written to /usr/share/elasticsearch/elastic-certificates.p12
……中间一大堆……
For client applications, you may only need to copy the CA certificate and
configure the client to trust this certificate.
检查一下
[root@1b2b7e0c0734 elasticsearch]# ls -ltr
total 576
-rw-r--r-- 1 elasticsearch root 8164 Feb 6 00:07 README.asciidoc
……一些ES自己的文件……
drwxrwxr-x 1 elasticsearch root 4096 Mar 24 16:34 config
-rw------- 1 root root 2527 Mar 28 15:45 elastic-stack-ca.p12
-rw------- 1 root root 3443 Mar 28 15:52 elastic-certificates.p12 <--- 多了一个它
- docker节点之间不能直接拷文件有点烦人（不是完全不可以，不过过程很曲折），只能先把这个认证文件拷到宿主机上，然后再从宿主机拷到其他节点里。不过我们可以直接通过宿主机挂载的方式，以宿主机为介质让所有节点共享这个文件。
- 先把文件拷到宿主机
docker cp node1:/usr/share/elasticsearch/elastic-certificates.p12 . - 在所有节点的挂载配置上加一行
- ./elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12
- 先把文件拷到宿主机
services:
node1:
volumes:
- ./elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12 // 加这行
- 这时就可以开启1.里面的两个配置，让集群正常启动。但是此时，我们通过kibana登陆的时候会要求输入账号密码。
- 我们来给集群创建一组密码，方便我们通过不同身份进行登陆和集群的操作。
(宿主机上登陆某个几点) docker exec -it node1 bash
[root@9a7aeeb9be4d elasticsearch]# ./bin/elasticsearch-setup-passwords auto // 自动创建所有用户及密钥，也可以通过手动方式指定
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
The passwords will be randomly generated and printed to the console.
Please confirm that you would like to continue [y/N]y
Changed password for user apm_system
PASSWORD apm_system = 9SMEwVztnQT3EkTPaQ7X
Changed password for user kibana
PASSWORD kibana = I28UJQgCoMUDM2SPjyu9
Changed password for user logstash_system
PASSWORD logstash_system = KNlDRpZpdSqFyaKjyiy2
Changed password for user beats_system
PASSWORD beats_system = U6vajAbRBI5RwX00CYuv
Changed password for user remote_monitoring_user
PASSWORD remote_monitoring_user = ez2eNRJty1ACp18cv5Wy
Changed password for user elastic
PASSWORD elastic = 5RgiAQSCvGyHZdW5EsYy
至此，我们就能够正常的启动3es+1kibana节点的docker集群了。
查看一下
// 启动集群
➜ ✗ docker-compose -f "docker-elasticsearch/docker-compose.yml" up -d --build
// 查看docker节点（们）的状态
➜ ✗ docker ps -as
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES SIZE
cc9527af7fee docker.elastic.co/kibana/kibana:7.6.0 "/usr/local/bin/dumb…" 5 hours ago Up 5 hours 0.0.0.0:5601->5601/tcp kibana 135MB (virtual 1.14GB)
0641c9015768 docker.elastic.co/elasticsearch/elasticsearch:7.6.0 "/usr/local/bin/dock…" 16 hours ago Up 16 hours 9200/tcp, 9300/tcp node3 3.7MB (virtual 794MB)
0b1f7b4ae9c1 docker.elastic.co/elasticsearch/elasticsearch:7.6.0 "/usr/local/bin/dock…" 16 hours ago Up 16 hours 0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp node1 6.15MB (virtual 796MB)
a165b7826a35 docker.elastic.co/elasticsearch/elasticsearch:7.6.0 "/usr/local/bin/dock…" 16 hours ago Up 16 hours 9200/tcp, 9300/tcp node2 4.66MB (virtual 795MB)
// 查看es集群状态
➜ ✗ curl http://elastic:5RgiAQSCvGyHZdW5EsYy@localhost:9200/ | jq .
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 534 100 534 0 0 38142 0 --:--:-- --:--:-- --:--:-- 38142
{
"name": "node1",
"cluster_name": "docker-cluster",
"cluster_uuid": "wZFXKEITRKWVg36vUHWgyQ",
"version": {
"number": "7.6.0",
"build_flavor": "default",
"build_type": "docker",
"build_hash": "7f634e9f44834fbc12724506cc1da681b0c3b1e3",
"build_date": "2020-02-06T00:09:00.449973Z",
"build_snapshot": false,
"lucene_version": "8.4.0",
"minimum_wire_compatibility_version": "6.8.0",
"minimum_index_compatibility_version": "6.0.0-beta1"
},
"tagline": "You Know, for Search"
}
#由于我们开启了密码验证，所以简单的通过localhost:9200来查看集群状态的话会报安全认证错误
➜ elasticsearch git:(7.6) ✗ curl http://localhost:9200/ | jq .
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 381 100 381 0 0 22411 0 --:--:-- --:--:-- --:--:-- 22411
{
"error": {
"root_cause": [
{
"type": "security_exception",
"reason": "missing authentication credentials for REST request [/]",
"header": {
"WWW-Authenticate": "Basic realm=\"security\" charset=\"UTF-8\""
}
}
],
"type": "security_exception",
"reason": "missing authentication credentials for REST request [/]",
"header": {
"WWW-Authenticate": "Basic realm=\"security\" charset=\"UTF-8\""
}
},
"status": 401
}
完整的docker-compose.yml文件
version: '3.6'
networks:
bigdata:
external: true
volumes:
esdata01:
driver: local
esdata02:
driver: local
esdata03:
driver: local
services:
node1:
image: docker.elastic.co/elasticsearch/elasticsearch:7.6.0
container_name: node1
environment:
- node.name=node1
- cluster.name=docker-cluster
- discovery.seed_hosts=node2,node3
- cluster.initial_master_nodes=node1
- xpack.security.enabled=true
- xpack.security.transport.ssl.enabled=true
- xpack.security.transport.ssl.keystore.type=PKCS12
- xpack.security.transport.ssl.verification_mode=certificate
- xpack.security.transport.ssl.keystore.path=elastic-certificates.p12
- xpack.security.transport.ssl.truststore.path=elastic-certificates.p12
- xpack.security.transport.ssl.truststore.type=PKCS12
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms2g -Xmx2g"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- esdata01:/usr/share/elasticsearch/data
- ./elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12
ports:
- 9200:9200
- 9300:9300
networks:
- bigdata
node2:
image: docker.elastic.co/elasticsearch/elasticsearch:7.6.0
container_name: node2
environment:
- node.name=node2
- cluster.name=docker-cluster
- discovery.seed_hosts=node1,node3
- cluster.initial_master_nodes=node1
- xpack.security.enabled=true
- xpack.security.transport.ssl.enabled=true
- xpack.security.transport.ssl.keystore.type=PKCS12
- xpack.security.transport.ssl.verification_mode=certificate
- xpack.security.transport.ssl.keystore.path=elastic-certificates.p12
- xpack.security.transport.ssl.truststore.path=elastic-certificates.p12
- xpack.security.transport.ssl.truststore.type=PKCS12
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms2g -Xmx2g"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- esdata02:/usr/share/elasticsearch/data
- ./elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12
networks:
- bigdata
node3:
image: docker.elastic.co/elasticsearch/elasticsearch:7.6.0
container_name: node3
environment:
- node.name=node3
- cluster.name=docker-cluster
- discovery.seed_hosts=node1,node2
- cluster.initial_master_nodes=node1
- xpack.security.enabled=true
- xpack.security.transport.ssl.enabled=true
- xpack.security.transport.ssl.keystore.type=PKCS12
- xpack.security.transport.ssl.verification_mode=certificate
- xpack.security.transport.ssl.keystore.path=elastic-certificates.p12
- xpack.security.transport.ssl.truststore.path=elastic-certificates.p12
- xpack.security.transport.ssl.truststore.type=PKCS12
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms2g -Xmx2g"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- esdata03:/usr/share/elasticsearch/data
- ./elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12
networks:
- bigdata
kibana:
image: docker.elastic.co/kibana/kibana:7.6.0
container_name: kibana
environment:
ELASTICSEARCH_HOSTS: http://node3:9200
XPACK_MONITORING_ENABLED: "true"
ELASTICSEARCH_USERNAME: kibana
ELASTICSEARCH_PASSWORD: I28UJQgCoMUDM2SPjyu9
SERVER_HOST: 0.0.0.0
depends_on:
- node1
- node2
- node3
external_links:
- node1
- node2
- node3
networks:
- bigdata
ports:
- 5601:5601
这里还有个坑点，我们开启了密码验证之后，在kibana的配置当中需要添加es的username和password。
ELASTICSEARCH_USERNAME: kibana
ELASTICSEARCH_PASSWORD: I28UJQgCoMUDM2SPjyu9
分别对应kibana.yml里面的
elasticsearch.username: kibana
elasticsearch.password: changeme
但是当我们通过web页面登陆的时候，输入的账号密码不是kibana，而是elastic这个账号的。
用kibana登陆的时候
这是因为kibana这个账号是给kibana节点自己做认证用的，不是给用户用的，我们需要用elastic这个账号登陆，这个才是人类管理员的账号。
// 不能用这俩
Changed password for user kibana
PASSWORD kibana = I28UJQgCoMUDM2SPjyu9
// 要用这俩
Changed password for user elastic
PASSWORD elastic = 5RgiAQSCvGyHZdW5EsYy
pls enjoy
重点总结
- 安装docker相关软件
- 关闭xpack认证启动集群
- 登陆任意一个es节点，创建认证文件
- 创建不同账户及密码
- 开启所有安全认证，挂载ca文件，启动集群
- 登陆成功并继续后续操作
参考文献：
Running the Elastic Stack on Docker
Setting up Elasticsearch and Kibana on Docker with X-Pack security enabled
Configuring Kibana
Install Elasticsearch with Docker
Security settings in Elasticsearch
Encrypting communications in Elasticsearch