在 Docker Swarm 上部署 Minio 集群

1 Swarm 集群初始化
准备四台服务器分别为 minio1、minio2、minio3、minio4，选择 minio1 为主节点，另外三个为工作节点。
主节点执行docker swarm init --advertise-addr 192.168.3.48

将红框中内容分别在三个工作节点执行
主节点执行docker node ls
如果出现类似下面的输出，就说明集群创建成功了。

2 部署 minio 服务
创建密钥(下文compose-secrets.yaml中写秘钥就不用了执行了)
echo “minio” | docker secret create access_key -
echo “12345678” | docker secret create secret_key -

绑定容器与节点
docker node update --label-add minio-node1=true minio1
docker node update --label-add minio-node2=true minio2
docker node update --label-add minio-node3=true minio3
docker node update --label-add minio-node4=true minio4

3 配置Stack compose 文件
version: ‘3.7’
services:
minio1:
image: minio/minio:latest
hostname: minio1
volumes:
- minio1-data1:/export1
- minio1-data2:/export2
ports:
- “9001:9000”
networks:
- minio_distributed
deploy:
restart_policy:
delay: 10s
max_attempts: 10
window: 60s
placement:
constraints:
- node.labels.minio-node1==true
command: server http://minio{1…4}/export{1…2} --console-address “:10000”
environment:
MINIO_ACCESS_KEY: minio
MINIO_SECRET_KEY: 12345678
MINIO_PROMETHEUS_AUTH_TYPE: public
MINIO_PROMETHEUS_JOB_ID: minio-job
healthcheck:
test: [“CMD”, “curl”, “-f”, “http://localhost:9000/minio/health/live”]
interval: 30s
timeout: 20s
retries: 3

minio2:
image: minio/minio:latest
hostname: minio2
volumes:
- minio2-data1:/export1
- minio2-data2:/export2
ports:
- “9002:9000”
networks:
- minio_distributed
deploy:
restart_policy:
delay: 10s
max_attempts: 10
window: 60s
placement:
constraints:
- node.labels.minio-node2==true
command: server http://minio{1…4}/export{1…2} --console-address “:10000”
environment:
MINIO_ACCESS_KEY: minio
MINIO_SECRET_KEY: 12345678
MINIO_PROMETHEUS_AUTH_TYPE: public
MINIO_PROMETHEUS_JOB_ID: minio-job
healthcheck:
test: [“CMD”, “curl”, “-f”, “http://localhost:9000/minio/health/live”]
interval: 30s
timeout: 20s
retries: 3

minio3:
image: minio/minio:latest
hostname: minio3
volumes:
- minio3-data1:/export1
- minio3-data2:/export2
ports:
- “9003:9000”
networks:
- minio_distributed
deploy:
restart_policy:
delay: 10s
max_attempts: 10
window: 60s
placement:
constraints:
- node.labels.minio-node3==true
command: server http://minio{1…4}/export{1…2} --console-address “:10000”
environment:
MINIO_ACCESS_KEY: minio
MINIO_SECRET_KEY: 12345678
MINIO_PROMETHEUS_AUTH_TYPE: public
MINIO_PROMETHEUS_JOB_ID: minio-job
healthcheck:
test: [“CMD”, “curl”, “-f”, “http://localhost:9000/minio/health/live”]
interval: 30s
timeout: 20s
retries: 3

minio4:
image: minio/minio:latest
hostname: minio4
volumes:
- minio4-data1:/export1
- minio4-data2:/export2
ports:
- “9004:9000”
networks:
- minio_distributed
deploy:
restart_policy:
delay: 10s
max_attempts: 10
window: 60s
placement:
constraints:
- node.labels.minio-node4==true
command: server http://minio{1…4}/export{1…2} --console-address “:10000”
environment:
MINIO_ACCESS_KEY: minio
MINIO_SECRET_KEY: 12345678
MINIO_PROMETHEUS_AUTH_TYPE: public
MINIO_PROMETHEUS_JOB_ID: minio-job
healthcheck:
test: [“CMD”, “curl”, “-f”, “http://localhost:9000/minio/health/live”]
interval: 30s
timeout: 20s
retries: 3
volumes:
minio1-data1:
minio1-data2:
minio2-data1:
minio2-data2:
minio3-data1:
minio3-data2:
minio4-data1:
minio4-data2:
networks:
minio_distributed:
driver: overlay
secrets:
secret_key:
external: true
access_key:
external: true
创建docker-compose-secrets.yaml配置文件
在docker-compose-secrets.yaml目录下执行
docker stack deploy --compose-file=docker-compose-secrets.yaml minio_stack

查看节点信息docker service ls

查看容器信息docker ps -a
如果不出意外，在四个节点上都会分别运行一个 docker 容器。通过 docker logs -f 可以查看日志，如果有类似下面的输出，表示启动成功：

4 删除集群
如果出现错误，想推导重来，那么可以通过下面这条命令删掉这个 stack：
docker stack rm minio_stack
集群会在每个节点逐个删除容器。如果删不掉，就只能手动 kill 了。创建的 volume 不会自动删除，需要在每个节点上手动执行下面的命令进行清除：
docker volume prune
docker swarm leave --force每个节点执行，然后就可以重新初始化swarm
补充：docker容器添加对外映射端口
将现有的容器打包成镜像，然后在使用新的镜像运行容器时重新指定要映射的端口
大概过程如下：
先停止现有容器
docker stop container-name
将容器commit成为一个镜像
docker commit container-name new-minio
用新镜像运行容器
docker run -it -d --name container-name -p p1:p1 -p p2:p2 new-minio

使用portainer可视化界面，分别在四个服务中添加minio console的端口