k8s二进制(ETCD的部署安装)

角色 ip 组件
k8s-master 192.168.11.169 kube-apiserver,kube-controller-manager,kube-scheduler,etcd
k8s-node1 192.168.11.164 kubelet,kube-proxy,docker,etcd
k8s-node2 192.168.11.166 kubelet,kube-proxy,docker,etcd

1、为etcd签发证书

1、证书的下载(任意机器上执行都可以)

#工作目录
mkdir -vp /usr/local/kubernetes/{etcd,cfssl}
#进入带cfssl证书目录下
mkdir -vp /opt/etcd/{bin,ssl,cfg}
cd /usr/local/kubernetes/cfssl

#github二进制包下载地址:https://github.com/cloudflare/cfssl/releases
#wget https://github.com/cloudflare/cfssl/releases/download/v1.6.1/cfssl_1.6.1_linux_amd64
#wget https://github.com/cloudflare/cfssl/releases/download/v1.6.1/cfssljson_1.6.1_linux_amd64
#wget https://github.com/cloudflare/cfssl/releases/download/v1.6.1/cfssl-certinfo_1.6.1_linux_amd64

# 生成证书
wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
# 利用Json生成证书
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
# 查看证书信息的工具
https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
#其他版本下载地址:https://github.com/cloudflare/cfssl/tags
chmod +x cfssl_linux-amd64 cfssljson_linux-amd64 cfssl-certinfo_linux-amd64
#\cp -R  cfssl_linux-amd64 cfssljson_linux-amd64 cfssl-certinfo_linux-amd64 /usr/local/bin/
#mv cfssl_linux-amd64 /usr/local/bin/cfssl
#mv cfssljson_linux-amd64 /usr/local/bin/cfssljson
#mv cfssl-certinfo_linux-amd64 /usr/bin/cfssl-certinfo
\cp -R cfssl_linux-amd64 /usr/local/bin/cfssl
\cp -R cfssljson_linux-amd64 /usr/local/bin/cfssljson
\cp -R cfssl-certinfo_linux-amd64 /usr/bin/cfssl-certinfo

#查看版本信息
cfssl version
Version: 1.2.0
Revision: dev
Runtime: go1.6

2、生成自签证书颁发机构(CA)

k8s二进制(ETCD的部署安装)_第1张图片

1、配置CA证书请求文件

cd /usr/local/kubernetes/etcd
#创建ca-csr.json
cat > ca-csr.json <<"EOF"
{
  "CN": "kubernetes",
  "key": {
      "algo": "rsa",
      "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "Beijing",
      "L": "Beijing",
      "O": "kubemsb",
      "OU": "CN"
    }
  ],
  "ca": {
          "expiry": "87600h"
  }
}
EOF

2、生成CA

cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
#查看是否生成成功
ls *pem ca*csr
#显示ca-key.pem ca.pem这2个文件即生成成功

3、配置ca证书策略

#创建ca-config.json
cat > ca-config.json << EOF
{
  "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "kubernetes": {
         "expiry": "87600h",
         "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ]
      }
    }
  }
}
EOF
  • server auth 表示client可以对使用该ca对server提供证书进行验证
  • client auth 表示server可以使用该ca对client提供的证书进行验证
    k8s二进制(ETCD的部署安装)_第2张图片

3、使用自签CA 签发Etcd HTTPS 证书

1、创建证书申请文件

cd /usr/local/kubernetes/etcd
cat > etcd-csr.json << EOF
{
	"CN": "etcd",
	"hosts": [
		"127.0.0.1",
		"192.168.11.169",
		"192.168.11.166",
		"192.168.11.164"
	],
	"key": {
		"algo": "rsa",
		"size": 2048
	},
	"names": [{
		"C": "CN",
		"ST": "Beijing",
		"L": "Beijing",
		"O": "kubemsb",
		"OU": "CN"
	}]
}
EOF

以上ip修改为自己集群环境的ip

2、生成证书

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes etcd-csr.json | cfssljson -bare etcd
#查看证书是否生成成功
ls etcd*pem

k8s二进制(ETCD的部署安装)_第3张图片

2、部署etcd集群

1、下载etcd

cd /usr/local/kubernetes/etcd/
#下载etcd文件
wget https://github.com/etcd-io/etcd/releases/download/v3.5.9/etcd-v3.5.9-linux-amd64.tar.gz
#wget https://github.com/etcd-io/etcd/releases/download/v3.5.2/etcd-v3.5.2-linux-amd64.tar.gz
#系统对应的版本https://github.com/etcd-io/etcd/releases/download/v3.5.7/etcd-v3.5.7-linux-amd64.tar.gz
#解压缩
tar zxvf etcd-v3.5.2-linux-amd64.tar.gz
#将下载下来的二进制可执行文件复制到etcd的工作目录中(用于复制到其他服务器上,部署使用)
\cp -R etcd-v3.5.2-linux-amd64/{etcd,etcdctl} /opt/etcd/bin

核心是复制etcd,etcdctl可执行文件到指定目录中,进行后续统一的远程传递。以及etcd集群的搭建

2、创建etcd.conf配置文件

cat > /opt/etcd/cfg/etcd.conf << EOF
#[Member]
ETCD_NAME="etcd-1"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
#填写当前节点的ip地址即可
ETCD_LISTEN_PEER_URLS="https://192.168.11.169:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.11.169:2379,http://127.0.0.1:2379"
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.11.169:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.11.169:2379"
#配置所有etcd集群对应的ip服务配置项
ETCD_INITIAL_CLUSTER="etcd-1=https://192.168.11.169:2380,etcd-2=https://192.168.11.166:2380,etcd-3=https://192.168.11.164:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF
  • 配置配置选项说明
ETCD_NAME:节点名称,集群中唯一
ETCD_DATA_DIR:数据目录
ETCD_LISTEN_PEER_URLS:集群通信监听地址
ETCD_LISTEN_CLIENT_URLS:客户端访问监听地址
ETCD_INITIAL_ADVERTISE_PEER_URLS:集群通告地址
ETCD_ADVERTISE_CLIENT_URLS:客户端通告地址
ETCD_INITIAL_CLUSTER:集群节点地址
ETCD_INITIAL_CLUSTER_TOKEN:集群Token
ETCD_INITIAL_CLUSTER_STATE:加入集群的当前状态,new 是新集群,existing 表示加入已有集群

3、systemd 管理etcd

cat > /usr/lib/systemd/system/etcd.service << EOF
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
EnvironmentFile=/opt/etcd/cfg/etcd.conf
ExecStart=/opt/etcd/bin/etcd \
--cert-file=/opt/etcd/ssl/etcd.pem \
--key-file=/opt/etcd/ssl/etcd-key.pem \
--peer-cert-file=/opt/etcd/ssl/etcd.pem \
--peer-key-file=/opt/etcd/ssl/etcd-key.pem \
--trusted-ca-file=/opt/etcd/ssl/ca.pem \
--peer-trusted-ca-file=/opt/etcd/ssl/ca.pem \
--peer-client-cert-auth \
--client-cert-auth \
--logger=zap
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF

4、拷贝生成的证书到ssl工作目录下

cd /usr/local/kubernetes/etcd
#复制ca办法机构和证书文件到etcd的ssl目录下准备生成etcd集群
\cp -R /usr/local/kubernetes/etcd/ca*.pem etcd*.pem  /opt/etcd/ssl/
cd /opt/etcd
tree .
[root@k8s-master1 etcd]# tree .
.
├── bin
│   ├── etcd
│   └── etcdctl
├── cfg
│   └── etcd.conf
└── ssl
    ├── ca-key.pem
    ├── ca.pem
    ├── etcd-key.pem
    └── etcd.pem

3 directories, 7 files
[root@k8s-master1 etcd]#

5、将节点etcd-1上的文件拷贝到节点2、3上

  • 说明
    1、 etcd-1上面的文件是已经生成好的,直接传输到需要部署etcd的服务器上即可
    2、一定记得修改对应的ip配置等信息
    3、节点etcd-2和etcd-3文件一直,需要更改为自己的ip地址

1、复制文件

#在master其他节点创建工作目录
#mkdir -vp /opt/etcd
#节点1复制到节点2上
scp -r /opt/etcd/* [email protected]:/opt/etcd
scp /usr/lib/systemd/system/etcd.service [email protected]:/usr/lib/systemd/system/
#节点1复制到节点3上
scp -r /opt/etcd/* [email protected]:/opt/etcd
scp /usr/lib/systemd/system/etcd.service [email protected]:/usr/lib/systemd/system/

k8s二进制(ETCD的部署安装)_第4张图片

2、修改对应的ip地址

vi /opt/etcd/cfg/etcd.conf
#[Member]
ETCD_NAME="etcd-1" # 修改此处,节点2 改为etcd-2,节点3 改为etcd-3
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
#将192.168.11.169修改为当前服务器的ip地址
ETCD_LISTEN_PEER_URLS="https://192.168.11.169:2380" # 修改此处为当前服务器IP
ETCD_LISTEN_CLIENT_URLS="https://192.168.11.169:2379" # 修改此处为当前服务器IP
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.11.169:2380" # 修改此处为当前服务器IP
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.11.169:2379" # 修改此处为当前服务器IP
ETCD_INITIAL_CLUSTER="etcd-1=https://192.168.11.169:2380,etcd-2=https://192.168.11.166:2380,etcd-3=https://192.168.11.164:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

6、启动并设置开机启动

节点都执行启动命令

systemctl daemon-reload
#启动etcd
systemctl start etcd
#开机启动
systemctl enable etcd
#查看启动状态
systemctl status etcd

k8s二进制(ETCD的部署安装)_第5张图片

7、记录错误

1、某一台服务一直启动不起起来

request sent was ignored (cluster ID mismatch: remote[9e5055697e

解决办法是:在配置的ETCD_DATA_DIR删除下面的文件

8、集群状态查看

  • 里面的ip由于做了很多次、ip修改为自己的正确ip

1、验证etcd集群状态

ETCDCTL_API=3 /opt/etcd/bin/etcdctl --write-out=table --cacert=/opt/etcd/ssl/ca.pem \
--cert=/opt/etcd/ssl/etcd.pem --key=/opt/etcd/ssl/etcd-key.pem \
--endpoints=https://192.168.11.173:2379,https://192.168.11.174:2379,https://192.168.11.175:2379 endpoint health

在这里插入图片描述

2、集群状态性能

ETCDCTL_API=3 /opt/etcd/bin/etcdctl --write-out=table --cacert=/opt/etcd/ssl/ca.pem \
--cert=/opt/etcd/ssl/etcd.pem --key=/opt/etcd/ssl/etcd-key.pem \
--endpoints=https://192.168.11.173:2379,https://192.168.11.174:2379,https://192.168.11.175:2379 check perf

在这里插入图片描述

3、集群状态成员列表

ETCDCTL_API=3 /opt/etcd/bin/etcdctl --write-out=table --cacert=/opt/etcd/ssl/ca.pem \
--cert=/opt/etcd/ssl/etcd.pem --key=/opt/etcd/ssl/etcd-key.pem \
--endpoints=https://192.168.11.173:2379,https://192.168.11.174:2379,https://192.168.11.175:2379 member list

k8s二进制(ETCD的部署安装)_第6张图片

4、查看集群状态(主从状态)

ETCDCTL_API=3 /opt/etcd/bin/etcdctl --write-out=table --cacert=/opt/etcd/ssl/ca.pem \
--cert=/opt/etcd/ssl/etcd.pem --key=/opt/etcd/ssl/etcd-key.pem \
--endpoints=https://192.168.11.173:2379,https://192.168.11.174:2379,https://192.168.11.175:2379 endpoint status

在这里插入图片描述

你可能感兴趣的:(kubernetes,etcd,容器)