es配置x-pack

环境背景：Ubuntu 20.04 LTS、es-7.7.0、docker-19.03.8、docker-compose-1.25.5

参考链接：https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-tls-docker.html

一、新建文件

1.1 在目录/data/operations下分别创建如下3个文件

1.1.1 .env文件内容

# Use an es_ prefix for all volumes and networks created by docker-compose
# COMPOSE_PROJECT_NAME=es
CERTS_DIR_ES=/usr/share/elasticsearch/config/certs
CERTS_DIR_KIBANA=/usr/share/kibana/config/certs
CERTS_DIR_LOGSTASH=/usr/share/logstash/config/certs
CERTS_DIR_FILEBEAT=/usr/share/filebeat/config/certs
ELASTIC_PASSWORD=123456
PWD=/data/operations

1.1.2 create-certs.yml文件内容

version: '3'

services:
  create_certs:
    container_name: create_certs
    image: docker.elastic.co/elasticsearch/elasticsearch:7.7.0
    command: >
      bash -c '
        if [[ ! -f /certs/bundle.zip ]]; then
          bin/elasticsearch-certutil cert --silent --pem --in config/instances.yml -out /certs/bundle.zip;
          unzip /certs/bundle.zip -d /certs; 
        fi;
        chown -R 1000:0 /certs
      '
    user: "0"
    # working_dir: /usr/share/elasticsearch
    volumes: 
      - ${PWD}/config/certs:/certs
      - ${PWD}/config/instances.yml:/usr/share/elasticsearch/config/instances.yml

1.1.3  es-docker-compose.yml文件内容

version: '3.8'

networks:
  es-shared:
    external:
      name: es-shared

services:
  es01:
    container_name: es01
    image: docker.elastic.co/elasticsearch/elasticsearch:7.7.0
    environment:
      ELASTIC_PASSWORD: ${ELASTIC_PASSWORD}
      CERTS_DIR_ES: ${CERTS_DIR_ES}
      ES_JAVA_OPTS: -Xms512m -Xmx512m
      # xpack.license.self_generated.type: trial
    volumes:
      - /data/operations/data/es01:/usr/share/elasticsearch/data
      - /data/operations/config/certs/es01:${CERTS_DIR_ES}/es01
      - /data/operations/config/certs/ca:${CERTS_DIR_ES}/ca
      - ${PWD}/config/es01.yml:/usr/share/elasticsearch/config/elasticsearch.yml
    ports:
      - 9200:9200
    networks:
      - es-shared
    healthcheck:
      test: curl --cacert ${CERTS_DIR_ES}/ca/ca.crt -s https://localhost:9200 >/dev/null; if [[ $$? == 52 ]]; then echo 0; else echo 1; fi
      interval: 30s
      timeout: 10s
      retries: 5

  es02:
    container_name: es02
    image: docker.elastic.co/elasticsearch/elasticsearch:7.7.0
    environment:
      ELASTIC_PASSWORD: ${ELASTIC_PASSWORD}
      CERTS_DIR_ES: ${CERTS_DIR_ES}
      ES_JAVA_OPTS: -Xms512m -Xmx512m
      # xpack.license.self_generated.type: trial
    volumes: 
      - /data/operations/data/es02:/usr/share/elasticsearch/data
      - /data/operations/config/certs/es02:${CERTS_DIR_ES}/es02
      - /data/operations/config/certs/ca:${CERTS_DIR_ES}/ca
      - ${PWD}/config/es02.yml:/usr/share/elasticsearch/config/elasticsearch.yml
    ports:
      - 9201:9200
    networks:
      - es-shared

 1.2 在目录/data/operations下分别创建如下3个文件

1.2.1 es01.yml文件内容

cluster.name: es
node.name: node-1
network.host: 0.0.0.0
http.port: 9200
discovery.seed_hosts: ["es01", "es02"]
cluster.initial_master_nodes: ["node-1", "node-2"]
xpack.security.enabled: true
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: ${CERTS_DIR_ES}/es01/es01.key
xpack.security.http.ssl.certificate_authorities: ${CERTS_DIR_ES}/ca/ca.crt
xpack.security.http.ssl.certificate: ${CERTS_DIR_ES}/es01/es01.crt
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.certificate_authorities: ${CERTS_DIR_ES}/ca/ca.crt
xpack.security.transport.ssl.certificate: ${CERTS_DIR_ES}/es01/es01.crt
xpack.security.transport.ssl.key: ${CERTS_DIR_ES}/es01/es01.key

1.2.1 es02.yml文件内容

cluster.name: es
node.name: node-2
network.host: 0.0.0.0
http.port: 9200
discovery.seed_hosts: ["es01", "es02"]
cluster.initial_master_nodes: ["node-1", "node-2"]
xpack.security.enabled: true
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: ${CERTS_DIR_ES}/es02/es02.key
xpack.security.http.ssl.certificate_authorities: ${CERTS_DIR_ES}/ca/ca.crt
xpack.security.http.ssl.certificate: ${CERTS_DIR_ES}/es02/es02.crt
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.certificate_authorities: ${CERTS_DIR_ES}/ca/ca.crt
xpack.security.transport.ssl.certificate: ${CERTS_DIR_ES}/es02/es02.crt
xpack.security.transport.ssl.key: ${CERTS_DIR_ES}/es02/es02.key

1.2.1 instances.yml文件内容

instances:
  - name: es01
    dns:
      - es01
      - localhost
    ip:
      - 127.0.0.1

  - name: es02
    dns:
      - es02
      - localhost
    ip:
      - 127.0.0.1

  - name: kibana01
    dns:
      - kibana01
      - localhost
    ip:
      - 127.0.0.1

  - name: logstash01
    dns:
      - logstash01
      - localhost
    ip:
      - 127.0.0.1

  - name: filebeat01
    dns:
      - filebeat01
      - localhost
    ip:
      - 127.0.0.1

  - name: metricbeat01
    dns:
      - metricbeat01
      - localhost
    ip:
      - 127.0.0.1

二、创建证书

2.1 切换到/data/operations目录下

docker-compose -f create-certs.yml run --rm create_certs

2.2 查看证书

三、创建并启动es容器

3.1 创建es集群并启动

docker-compose -f es-docker-compose.yml up -d

3.2 查看结果

docker ps -a | egrep "es01|es02"

3.2.1 如果容器启动失败

3.2.1 查看容器日志

docker logs es01

 3.2.3 日志内容

Created elasticsearch keystore in /usr/share/elasticsearch/config/elasticsearch.keystore
{"type": "server", "timestamp": "2020-07-14T02:31:15,047Z", "level": "ERROR", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.name": "es", "node.name": "node-1", "message": "uncaught exception in thread [main]", 
"stacktrace": ["org.elasticsearch.bootstrap.StartupException: ElasticsearchException[failed to bind service]; nested: AccessDeniedException[/usr/share/elasticsearch/data/nodes];",
"at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:174) ~[elasticsearch-7.7.0.jar:7.7.0]",
"at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) ~[elasticsearch-7.7.0.jar:7.7.0]",
"at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.7.0.jar:7.7.0]",
"at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127) ~[elasticsearch-cli-7.7.0.jar:7.7.0]",
"at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.7.0.jar:7.7.0]",
"at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) ~[elasticsearch-7.7.0.jar:7.7.0]",
uncaught exception in thread [main]
"at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.7.0.jar:7.7.0]",
"Caused by: org.elasticsearch.ElasticsearchException: failed to bind service",
"at org.elasticsearch.node.Node.(Node.java:638) ~[elasticsearch-7.7.0.jar:7.7.0]",
"at org.elasticsearch.node.Node.(Node.java:264) ~[elasticsearch-7.7.0.jar:7.7.0]",
"at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:227) ~[elasticsearch-7.7.0.jar:7.7.0]",
"at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:227) ~[elasticsearch-7.7.0.jar:7.7.0]",
"at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393) ~[elasticsearch-7.7.0.jar:7.7.0]",
"at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.7.0.jar:7.7.0]",
"... 6 more",
"Caused by: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes",
"at sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) ~[?:?]",
"at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]",
"at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) ~[?:?]",
"at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:389) ~[?:?]",
"at java.nio.file.Files.createDirectory(Files.java:694) ~[?:?]",
"at java.nio.file.Files.createAndCheckIsDirectory(Files.java:801) ~[?:?]",
"at java.nio.file.Files.createDirectories(Files.java:787) ~[?:?]",
"at org.elasticsearch.env.NodeEnvironment.lambda$new$0(NodeEnvironment.java:274) ~[elasticsearch-7.7.0.jar:7.7.0]",
"at org.elasticsearch.env.NodeEnvironment$NodeLock.(NodeEnvironment.java:211) ~[elasticsearch-7.7.0.jar:7.7.0]",
"at org.elasticsearch.env.NodeEnvironment.(NodeEnvironment.java:271) ~[elasticsearch-7.7.0.jar:7.7.0]",
"at org.elasticsearch.node.Node.(Node.java:284) ~[elasticsearch-7.7.0.jar:7.7.0]",
"at org.elasticsearch.node.Node.(Node.java:264) ~[elasticsearch-7.7.0.jar:7.7.0]",
"at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:227) ~[elasticsearch-7.7.0.jar:7.7.0]",
"at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:227) ~[elasticsearch-7.7.0.jar:7.7.0]",
"at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393) ~[elasticsearch-7.7.0.jar:7.7.0]",
"at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.7.0.jar:7.7.0]",
"... 6 more"] }
ElasticsearchException[failed to bind service]; nested: AccessDeniedException[/usr/share/elasticsearch/data/nodes];
Likely root cause: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes
	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
	at java.base/sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:389)
	at java.base/java.nio.file.Files.createDirectory(Files.java:694)
	at java.base/java.nio.file.Files.createAndCheckIsDirectory(Files.java:801)
	at java.base/java.nio.file.Files.createDirectories(Files.java:787)
	at org.elasticsearch.env.NodeEnvironment.lambda$new$0(NodeEnvironment.java:274)
	at org.elasticsearch.env.NodeEnvironment$NodeLock.(NodeEnvironment.java:211)
	at org.elasticsearch.env.NodeEnvironment.(NodeEnvironment.java:271)
	at org.elasticsearch.node.Node.(Node.java:284)
	at org.elasticsearch.node.Node.(Node.java:264)
	at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:227)
	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:227)
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393)
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170)
	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161)
	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127)
	at org.elasticsearch.cli.Command.main(Command.java:90)
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126)
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92)
For complete error details, refer to the log at /usr/share/elasticsearch/logs/es.log

从日志内容可以看出访问数据节点权限被禁止

3.2.4 打开另一个终端，切换到数据挂载目录/data/operations/data，查看目录权限

cd /data/operations/data
ls -lh

从结果可以看出，当前用户为ubuntu，对es01、es02目录没有可写的权限，所以当前用户创建容器时，es不能创建数据节点

3.2.5 修改目录权限或拥有者

sudo chown ubuntu es01 es02
ls -lh

3.3 重启创建容器并启

3.3.1 查看容器启动结果 

 3.3.2 查看数据挂载目录结果

四、查看结果

4.1 打开浏览器，查看http请求：http://192.168.1.121:9200/ 

4.2 查看https请求：https://192.168.1.121:9200/ 

4.2.1 选择高级

 4.2.2 选择继续前往

4.2.3 输入账号密码进行登录

账号：elastic
密码：123456

4.2.4 登录成功后结果

4.3 使用引导密码通过SSL/TLS访问Elasticsearch API

4.3.1 通过容器连接

docker run --rm -v /data/operations/config/certs:/certs --network=es-shared docker.elastic.co/elasticsearch/elasticsearch:7.7.0 curl --cacert /certs/ca/ca.crt -u elastic:123456 https://es01:9200

4.3.2 通过ip直接连接

curl --cacert /data/operations/config/certs/ca/ca.crt -u elastic:123456 https://localhost:9200
curl --cacert /data/operations/config/certs/ca/ca.crt -u elastic:123456 https://127.0.0.1:9200

 五、设置内置账户密码

5.1 生成随机密码

docker exec es01 /bin/bash -c "bin/elasticsearch-setup-passwords auto --batch --url https://localhost:9200"

5.2 手动指定密码

5.2.1 首先进入容器内部

docker exec -it es01 bash

5.2.2 设置密码 

bin/elasticsearch-setup-passwords interactive --url https://localhost:9200