docker篇-(docker-compose安装etcd高可用集群)

构建etcd镜像

这里我安装的etcd集群使用的是3.4.9版本的
1.下载etcd安装包,并解压

# cd到当前用户所在目录并创建etcd然后到etcd目录
cd ~ && mkdir etcd && cd etcd
# 下载etcd二进制压缩包并解压
wget https://repo.huaweicloud.com/etcd/v3.4.9/etcd-v3.4.9-linux-amd64.tar.gz && tar zxvf etcd-v3.4.9-linux-amd64.tar.gz
# 删除下载的etcd压缩包
rm -rf etcd-v3.4.9-linux-amd64.tar.gz
# 创建bin目录，并将etcd解压出来的二进制文件复制到bin目录
mkdir bin && mv etcd-v3.4.9-linux-amd64/{etcd,etcdctl} bin/
# 删除解压出来的etcd目录
rm -rf etcd-v3.4.9-linux-amd64/

2.编写Dockerfile文件

cat > Dockerfile <<EOF
FROM alpine
MAINTAINER [email protected]
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories \\    
	&& apk update \\    
	&& apk add --no-cache tzdata \\
	&& cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
ENV TZ Asia/Shanghai
VOLUME /work
WORKDIR /word
ADD bin/etcd /usr/sbin/etcd
ADD bin/etcdctl /usr/sbin/etcdctl
EXPOSE 2379 2380
CMD etcd
EOF

3.构建etcd镜像，启动并测试是否能够运行

# 构建镜像
docker build -t etcd:3.4.9-alpine .
# 启动etcd
docker run --rm -ti etcd:3.4.9-alpine

启动成功同如下页面

生成ssl证书

安装cfssl证书生成工具

wget -O /usr/local/sbin/certinfo https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 && chmod +x /usr/local/sbin/certinfo
wget -O /usr/local/sbin/cfssl https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 && chmod +x /usr/local/sbin/cfssl
wget -O /usr/local/sbin/cfssljson https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 && chmod +x /usr/local/sbin/cfssljson

编写生成证书的配置文件

1.创建证书文件存放的目录

# 后续生成的证书都会在ssl目录
mkdir ssl && cd ssl

2.目录结构如下

3.打印默认证书配置文件

cfssl print-defaults config > ca-config.json

4.修改ca-config.json中的配置

cat > ca-config.json <<EOF
{
    "signing": {
        "default": {
            "expiry": "876000h"
        },
        "profiles": {
            "etcd": {
                "expiry": "876000h",
                "usages": [
                    "signing",
                    "key encipherment",
                    "server auth",
                    "client auth"
                ]
            }
        }
    }
}
EOF

编写ca根证书配置文件

1.打印默认ca证书配置

cfssl print-defaults csr > ca-csr.json

2.修改ca-csr.json配置文件

cat > ca-csr.json <<EOF
{
    "CN": "www.etcd.com.cn",
    "key": {
        "algo": "ecdsa",
        "size": 256
    },
    "names": [
        {
            "City": "CN",
            "Country": "ChengDu",
            "Author": "Lhstack"
        }
    ]
}
EOF

3.生成ca-key.pem,ca.pem文件

cfssl gencert -initca ca-csr.json | cfssljson -bare ca

编写服务端认证证书配置文件

1.打印默认server端证书配置

cfssl print-defaults csr > server-csr.json

2.修改server-csr.json配置文件
hosts为etcd每一台集群的ip地址，如果后续要扩容，可以多预留几个ip

cat > server-csr.json <<EOF
{
    "CN": "www.etcd.com.cn",
    "hosts":[
    	"10.100.110.80",
    	"10.100.110.81",
    	"10.100.110.82"
    ],
    "key": {
        "algo": "ecdsa",
        "size": 256
    },
    "names": [
        {
            "City": "CN",
            "Country": "ChengDu",
            "Author": "Lhstack"
        }
    ]
}
EOF

3.生成server端认证证书server.pem,server-key.pem

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=etcd server-csr.json | cfssljson -bare server

编写客户端证书配置

1.打印默认server端证书配置

cfssl print-defaults csr > client-csr.json

2.修改client-csr.json配置文件

cat > client-csr.json <<EOF
{
    "CN": "www.etcd.com.cn",
    "key": {
        "algo": "ecdsa",
        "size": 256
    },
    "names": [
        {
            "City": "CN",
            "Country": "ChengDu",
            "Author": "Lhstack"
        }
    ]
}
EOF

3.生成客户端端认证证书client.pem,client-key.pem

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=etcd client-csr.json | cfssljson -bare client

编写docker-compose.yml文件

返回上级目录

cd ..

docker-compose.yml如下

version: '3'
services:
  etcd-1:
    image: etcd:3.4.9-alpine
    build: .
    container_name: etcd-1
    volumes:
    - ./ssl:/opt/etcd/ssl
    - ./data/etcd-1:/opt/etcd/data
    command:
    - sh
    - -c
    - |
      etcd -name etcd-1  --cert-file=/opt/etcd/ssl/server.pem \
      --listen-peer-urls https://10.100.110.80:2380 --listen-client-urls https://10.100.110.80:2379,http://10.100.110.80:2379 \
      --key-file=/opt/etcd/ssl/server-key.pem --peer-cert-file=/opt/etcd/ssl/server.pem \
      --peer-key-file=/opt/etcd/ssl/server-key.pem --initial-advertise-peer-urls https://10.100.110.80:2380 \
      --advertise-client-urls https://10.100.110.80:2379 --initial-cluster-token etcd-cluster \
      --trusted-ca-file=/opt/etcd/ssl/ca.pem --peer-trusted-ca-file=/opt/etcd/ssl/ca.pem \
      --initial-cluster etcd-1=https://10.100.110.80:2380,etcd-2=https://10.100.110.81:2380,etcd-3=https://10.100.110.82:2380 \
      --enable-v2=true \
      --data-dir=/opt/etcd/data
      --initial-cluster-state new
    networks:
      etcd:
        ipv4_address: "10.100.110.80"
  etcd-2:
    image: etcd:3.4.9-alpine
    container_name: etcd-2
    volumes:
    - ./ssl:/opt/etcd/ssl
    - ./data/etcd-2:/opt/etcd/data
    command:
    - sh
    - -c
    - |
      etcd -name etcd-2  --cert-file=/opt/etcd/ssl/server.pem \
      --listen-peer-urls https://10.100.110.81:2380 --listen-client-urls https://10.100.110.81:2379,http://10.100.110.81:2379 \
      --key-file=/opt/etcd/ssl/server-key.pem --peer-cert-file=/opt/etcd/ssl/server.pem \
      --peer-key-file=/opt/etcd/ssl/server-key.pem --initial-advertise-peer-urls https://10.100.110.81:2380 \
      --advertise-client-urls https://10.100.110.81:2379 --initial-cluster-token etcd-cluster \
      --trusted-ca-file=/opt/etcd/ssl/ca.pem --peer-trusted-ca-file=/opt/etcd/ssl/ca.pem \
      --initial-cluster etcd-1=https://10.100.110.80:2380,etcd-2=https://10.100.110.81:2380,etcd-3=https://10.100.110.82:2380 \
      --enable-v2=true \
      --data-dir=/opt/etcd/data
      --initial-cluster-state new
    networks:
      etcd:
        ipv4_address: "10.100.110.81"
  etcd-3:
    image: etcd:3.4.9-alpine
    container_name: etcd-3
    volumes:
    - ./ssl:/opt/etcd/ssl
    - ./data/etcd-3:/opt/etcd/data
    command:
    - sh
    - -c
    - |
      etcd -name etcd-3  --cert-file=/opt/etcd/ssl/server.pem \
      --listen-peer-urls https://10.100.110.82:2380 --listen-client-urls https://10.100.110.82:2379,http://10.100.110.82:2379 \
      --key-file=/opt/etcd/ssl/server-key.pem --peer-cert-file=/opt/etcd/ssl/server.pem \
      --peer-key-file=/opt/etcd/ssl/server-key.pem --initial-advertise-peer-urls https://10.100.110.82:2380 \
      --advertise-client-urls https://10.100.110.82:2379 --initial-cluster-token etcd-cluster \
      --trusted-ca-file=/opt/etcd/ssl/ca.pem --peer-trusted-ca-file=/opt/etcd/ssl/ca.pem \
      --initial-cluster etcd-1=https://10.100.110.80:2380,etcd-2=https://10.100.110.81:2380,etcd-3=https://10.100.110.82:2380 \
      --enable-v2=true \
      --data-dir=/opt/etcd/data
      --initial-cluster-state new
    networks:
      etcd:
        ipv4_address: "10.100.110.82"
networks:
  etcd:
    driver: bridge
    ipam:
      driver: default
      config:
      - subnet: 10.100.0.0/16

运行etcd集群

目录结构如下

执行docker-compose

docker-compose up -d

运行结果如下

查看集群状态

进入任意一个etcd集群节点

docker exec -ti etcd-1 sh

查看集群状态

etcdctl --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/client.pem --key=/opt/etcd/ssl/client-key.pem --endpoints="https://10.100.110.80:2379,https://10.100.110.81:2379,https://10.100.110.82:2379" endpoint status -w table

将etcdctl封装成脚本使用

cat > etcdctl.sh << EOF
#!/bin/sh
SSL_DIR=/opt/etcd/ssl
ENDPOINTS="https://10.100.110.80:2379,https://10.100.110.81:2379,https://10.100.110.82:2379"
etcdctl --cacert=\$SSL_DIR/ca.pem --cert=\$SSL_DIR/client.pem --key=\$SSL_DIR/client-key.pem --endpoints=\$ENDPOINTS \$@
EOF

使用脚本查看集群状态

sh etcdctl.sh endpoint status -w table

使用脚本添加数据

sh etcdctl.sh put msg hello

使用脚本获取数据

sh etcdctl.sh get msg

使用脚本删除数据

sh etcdctl.sh del msg