本次实验选择5台主机,3台作为master主机,2台作为node节点
节点ip | OS版本 | hostname -f | 安装软件 |
---|---|---|---|
192.168.0.1 | RHEL7.4 | k8s-master01 | docker,etcd,flanneld,kube-apiserver,kube-controller-manager,kube-scheduler |
192.168.0.2 | RHEL7.4 | k8s-master02 | docker,etcd,flanneld,kube-apiserver,kube-controller-manager,kube-scheduler |
192.168.0.3 | RHEL7.4 | k8s-master03 | docker,etcd,flanneld,kube-apiserver,kube-controller-manager,kube-scheduler |
192.168.0.4 | RHEL7.4 | k8s-node01 | docker,flanneld,kubelet,kube-proxy |
192.168.0.5 | RHEL7.4 | k8s-node02 | docker,flanneld,kubelet,kube-proxy |
kubernetes master 节点包含的组件:
目前这三个组件需要部署在同一台机器上。
# wget https://dl.k8s.io/v1.15.3/kubernetes-server-linux-amd64.tar.gz
# tar xf kubernetes-server-linux-amd64.tar.gz# cd kubernetes/server/bin/
# cp kubeadm kube-apiserver kube-controller-manager kubectl kube-scheduler /k8s/kubernetes/bin/
创建kubernetes 证书签名请求:
cat > kubernetes-csr.json <
{
"CN": "kubernetes",
"hosts": [
"127.0.0.1",
"192.168.0.1",
"192.168.0.2",
"192.168.0.3",
"101.254.0.1",
"kubernetes",
"kubernetes.default",
"kubernetes.default.svc",
"kubernetes.default.svc.cluster",
"kubernetes.default.svc.cluster.local"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "BeiJing",
"L": "BeiJing",
"O": "k8s",
"OU": "System"
}
]
}
EOF
# cfssl gencert -ca=/k8s/kubernetes/ssl/ca.pem -ca-key=/k8s/kubernetes/ssl/ca-key.pem -config=/k8s/kubernetes/ssl/ca-config.json -profile=kubernetes kubernetes-csr.json | cfssljson -bare kubernetes
# ls kub*
kubernetes.csr kubernetes-csr.json kubernetes-key.pem kubernetes.pem# cp kubernetes*.pem /k8s/kubernetes/ssl/
kubelet 首次启动时向kube-apiserver 发送TLS Bootstrapping 请求,kube-apiserver 验证请求中的token 是否与它配置的token.csv 一致,如果一致则自动为kubelet 生成证书和密钥。
TLS Bootstrapping 使用的Token可以使用命令生成
# head -c 16 /dev/urandom | od -An -t x | tr -d ' '
ef502f26a00ac235b04977cde1dc9916cat << EOF > /k8s/kubernetes/cfg/token.csv
ef502f26a00ac235b04977cde1dc9916,kubelet-bootstrap,10001,"system:kubelet-bootstrap"
EOF
cat << EOF > /k8s/kubernetes/cfg/kube-apiserver
KUBE_APISERVER_OPTS="--logtostderr=true \
--v=4 \
--etcd-servers=https://192.168.0.3:2379,https://192.168.0.2:2379,https://192.168.0.1:2379 \
--bind-address=192.168.0.1 \
--secure-port=6443 \
--advertise-address=192.168.0.1 \
--allow-privileged=true \
--service-cluster-ip-range=101.254.0.0/16 \
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota,NodeRestriction \
--authorization-mode=RBAC,Node \
--enable-bootstrap-token-auth \
--token-auth-file=/k8s/kubernetes/cfg/token.csv \
--service-node-port-range=30000-50000 \
--tls-cert-file=/k8s/kubernetes/ssl/kubernetes.pem \
--tls-private-key-file=/k8s/kubernetes/ssl/kubernetes-key.pem \
--client-ca-file=/k8s/kubernetes/ssl/ca.pem \
--service-account-key-file=/k8s/kubernetes/ssl/ca-key.pem \
--etcd-cafile=/k8s/kubernetes/ssl/ca.pem \
--etcd-certfile=/k8s/kubernetes/ssl/kubernetes.pem \
--etcd-keyfile=/k8s/kubernetes/ssl/kubernetes-key.pem"
EOF
提示:其他master节点需要将红字部分替换
cat << EOF > /lib/systemd/system/kube-apiserver.service
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/kubernetes/kubernetes[Service]
EnvironmentFile=-/k8s/kubernetes/cfg/kube-apiserver
ExecStart=/k8s/kubernetes/bin/kube-apiserver \$KUBE_APISERVER_OPTS
Restart=on-failure[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable kube-apiserver
systemctl start kube-apiserver
systemctl status kube-apiserver
kubectl create clusterrolebinding kube-apiserver:kubelet-apis --clusterrole=system:kubelet-api-admin --user kubernetes
cat << EOF > /k8s/kubernetes/cfg/kube-controller-manager
KUBE_CONTROLLER_MANAGER_OPTS="--logtostderr=true \
--v=4 \
--master=127.0.0.1:8080 \
--leader-elect=true \
--address=127.0.0.1 \
--service-cluster-ip-range=101.254.0.0/24 \
--cluster-cidr=100.100.0.0/16 \
--cluster-name=kubernetes \
--cluster-signing-cert-file=/k8s/kubernetes/ssl/ca.pem \
--cluster-signing-key-file=/k8s/kubernetes/ssl/ca-key.pem \
--root-ca-file=/k8s/kubernetes/ssl/ca.pem \
--service-account-private-key-file=/k8s/kubernetes/ssl/ca-key.pem"
EOF
提示:
--insecure-port=0,
关闭监听非安全端口(8080),则需要单独为kube-controller-manager创建证书。cat << EOF > /lib/systemd/system/kube-controller-manager.service
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/kubernetes/kubernetes[Service]
EnvironmentFile=-/k8s/kubernetes/cfg/kube-controller-manager
ExecStart=/k8s/kubernetes/bin/kube-controller-manager \$KUBE_CONTROLLER_MANAGER_OPTS
Restart=on-failure[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable kube-controller-manager
systemctl restart kube-controller-manager
systemctl status kube-controller-manager
cat << EOF > /k8s/kubernetes/cfg/kube-scheduler
KUBE_SCHEDULER_OPTS="--logtostderr=true --v=4 --master=127.0.0.1:8080 --leader-elect=true"
EOF
cat << EOF > /lib/systemd/system/kube-scheduler.service
[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/kubernetes/kubernetes[Service]
EnvironmentFile=-/k8s/kubernetes/cfg/kube-scheduler
ExecStart=/k8s/kubernetes/bin/kube-scheduler \$KUBE_SCHEDULER_OPTS
Restart=on-failure[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable kube-scheduler.service
systemctl restart kube-scheduler.service
systemctl status kube-scheduler.service
将可执行文件添加到 PATH 变量中
echo "export PATH=$PATH:/k8s/kubernetes/bin/" >>/etc/profile
source /etc/profile
#查看master服务状态
# kubectl get cs
NAME STATUS MESSAGE ERROR
controller-manager Healthy ok
scheduler Healthy ok
etcd-1 Healthy {"health":"true"}
etcd-2 Healthy {"health":"true"}
etcd-0 Healthy {"health":"true"}
scp -r /k8s/kubernetes/bin/* 192.168.0.2:/k8s/kubernetes/bin/
scp -r /k8s/kubernetes/bin/* 192.168.0.3:/k8s/kubernetes/bin/
scp -r /k8s/kubernetes/ssl/* 192.168.0.2:/k8s/kubernetes/ssl/
scp -r /k8s/kubernetes/ssl/* 192.168.0.3:/k8s/kubernetes/ssl/
scp -r /k8s/kubernetes/cfg/* 192.168.0.2:/k8s/kubernetes/cfg/
scp -r /k8s/kubernetes/cfg/* 192.168.0.3:/k8s/kubernetes/cfg/
scp /lib/systemd/system/kube-* 192.168.0.2:/lib/systemd/system/
scp /lib/systemd/system/kube-* 192.168.0.3:/lib/systemd/system/