5分钟用kubeadm部署k8s

5分钟用kubeadm部署k8s

玩了两年rke，想试试原生k8s，没想到部署起来还挺容易的，直接上干货，有问题留言。

配置主机

systemctl disable firewalld
systemctl stop firewalld
setenforce 0
sed -i 's/^SELINUX=.\+$/SELINUX=disable/g' /etc/sysconfig/selinux
swapoff -a
sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

安装依赖

kubeadm kubelet kubectl

cat << EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernets Repository
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
EOF

yum install --downloadonly --downloaddir=kubelet kubelet --disableexcludes=kubernetes
yum install --downloadonly --downloaddir=kubeadm kubeadm --disableexcludes=kubernetes
yum install --downloadonly --downloaddir=kubectl kubectl --disableexcludes=kubernetes

yum localinstall -y kubelet/* kubeadm/* kubectl/*

docker

# vi /etc/yum.repos.d/docker-ce.repo
[docker-ce-stable]
name=Docker CE Stable - $basearch
baseurl=https://download.docker.com/linux/centos/7/$basearch/stable
enabled=1
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg

[docker-ce-stable-debuginfo]
name=Docker CE Stable - Debuginfo $basearch
baseurl=https://download.docker.com/linux/centos/7/debug-$basearch/stable
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg

[docker-ce-stable-source]
name=Docker CE Stable - Sources
baseurl=https://download.docker.com/linux/centos/7/source/stable
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg

[docker-ce-edge]
name=Docker CE Edge - $basearch
baseurl=https://download.docker.com/linux/centos/7/$basearch/edge
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg

[docker-ce-edge-debuginfo]
name=Docker CE Edge - Debuginfo $basearch
baseurl=https://download.docker.com/linux/centos/7/debug-$basearch/edge
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg

[docker-ce-edge-source]
name=Docker CE Edge - Sources
baseurl=https://download.docker.com/linux/centos/7/source/edge
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg

[docker-ce-test]
name=Docker CE Test - $basearch
baseurl=https://download.docker.com/linux/centos/7/$basearch/test
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg

[docker-ce-test-debuginfo]
name=Docker CE Test - Debuginfo $basearch
baseurl=https://download.docker.com/linux/centos/7/debug-$basearch/test
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg

[docker-ce-test-source]
name=Docker CE Test - Sources
baseurl=https://download.docker.com/linux/centos/7/source/test
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg

[docker-ce-nightly]
name=Docker CE Nightly - $basearch
baseurl=https://download.docker.com/linux/centos/7/$basearch/nightly
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg

[docker-ce-nightly-debuginfo]
name=Docker CE Nightly - Debuginfo $basearch
baseurl=https://download.docker.com/linux/centos/7/debug-$basearch/nightly
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg

[docker-ce-nightly-source]
name=Docker CE Nightly - Sources
baseurl=https://download.docker.com/linux/centos/7/source/nightly
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg

yum install --downloadonly --downloaddir=docker-ce docker-ce
yum localinstall -y docker-ce/*

mkdir -p /etc/docker
cat << EOF > /etc/docker/daemon.json
{
     
    "registry-mirrors": ["https://registry.docker-cn.com"]
}
EOF

systemctl enable docker
systemctl restart docker
systemctl enable kubelet
systemctl restart kubelet

配置kubeadm并拉起k8s

kubeadm config print init-defaults > init.default.yaml
kubeadm config images list > images.list
sed -i 's/^imageRepository: .\+$/imageRepository: docker.io\/aiotceo/g' init.default.yaml
kubeadm config images pull --config=init.default.yaml

docker pull bantianyinshi/etcd:3.4.13-0
docker tag bantianyinshi/etcd:3.4.13-0 docker.io/aiotceo/etcd:3.4.13-0
docker pull docker.io/aiotceo/kube-apiserver:v1.20.1
docker pull docker.io/aiotceo/kube-controller-manager:v1.20.1
docker pull docker.io/aiotceo/kube-scheduler:v1.20.1
docker pull docker.io/aiotceo/kube-proxy:v1.20.1
docker pull docker.io/aiotceo/pause:3.2
docker pull docker.io/aiotceo/coredns:1.7.0

IP=192.168.103.181
sed -i 's/^  advertiseAddress: .\+$/  advertiseAddress: '"$IP"'/g' init.default.yaml
kubeadm init --config=init.default.yaml --v=5

部署成功后会有如下输出，提示如何使用kubectl，如何添加Node，如何添加网络插件，下面一步步来。

使用kubectl

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubectl get all --all-namespaces

添加Node

NODE_IP=192.168.103.176
scp -r docker-ce kubelet kubeadm $NODE_IP:/root/

ssh $NODE_IP <<EOF
yum localinstall -y /root/docker-ce/* /root/kubelet/* /root/kubeadm/*
mkdir -p /etc/docker
cat << EOFF > /etc/docker/daemon.json
{
    "insecure-registries":["cp-registry:5005","harbor.t2cp.com"],
    "registry-mirrors": ["https://8m0vweth.mirror.aliyuncs.com","https://registry.docker-cn.com"],
    "exec-opts": ["native.cgroupdriver=systemd"],
    "log-driver":"json-file",
    "log-opts": {"max-size":"100m", "max-file":"3"},
    "storage-driver": "overlay2",
    "storage-opts": ["overlay2.override_kernel_check=true"]
}
EOFF
systemctl enable docker
systemctl restart docker
systemctl enable kubelet
systemctl restart kubelet
ls
EOF

ssh $NODE_IP <<EOF
kubeadm join 192.168.103.181:6443 --token abcdef.0123456789abcdef \
    --discovery-token-ca-cert-hash sha256:b8d32117baab12e039ef8d784edc610636981b100c1fb467cb89655f27a2226d
EOF

添加网络插件

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"

到这就算部署完成了，如果前面部署失败了，需要清理环境后重新部署

清理

systemctl stop kubelet
docker ps -aq  | xargs -n1 -i{
     } sh -c " docker stop {} && docker rm -v {}"
docker volume rm $(sudo docker volume ls -q)

mount | grep '/var/lib/kubelet'| awk '{print $3}'|xargs umount
rm -rf /etc/kubernetes \
  /var/lib/etcd \
  /var/lib/kubelet  \
  /var/log/containers \
  /var/log/pods
for mount in $(mount | grep tmpfs | grep '/var/lib/kubelet' | awk '{ print $3 }') /var/lib/kubelet /var/lib/rancher; do umount $mount; done
rm -f /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db

sudo systemctl restart containerd
sudo systemctl restart docker

排障

这个是我在部署中遇到的问题以及如何解决

etcd容器报错: pkg/flags: conflicting environment variable “ETCD_ADVERTISE_CLIENT_URLS” is shadowed by corresponding command-line flag (either unset environment variable or disable flag)

原因是bitnami/etcd:3.4.13在原来的k8s.gcr.io/etcd:3.4.13-0基础上添加了一些环境变量，kubeadm拉起etcd时又以参数方式传入导致跟环境变量冲突，换用另一个etcd镜像即可。

参考引用

• etcd报错原因
• bitnami/etcd:3.4.13镜像中的环境变量
• 安装weave网络插件