K8S的创建
- 下载安装包：
a) ALI: https://developer.aliyun.com/mirror/
- 配置docker的yum仓库
a) 方法一：
cd /etc/yum.repos.d/
b) 下载安装包：
wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
c) 安装必要的一些系统工具
yum install -y yum-utils device-mapper-persistent-data lvm2
d) 添加软件源信息
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
e) 更新并安装Docker-CE
重新加载：
systemctl daemon-reload
yum -y install docker-ce
f) 开启Docker服务
service docker start -
配置k8s的yum仓库：
cat </etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
关闭SELUNUX
setenforce 0 - 生产yum源：
更新缓存
yum makecache
复制两yum源至其它两台主机notea.silence.com和noteb.silence.com
修改主机名映射：
vim /etc/hosts
10.206.35.235 note.silence.com
10.206.35.233 noteb.silence.com
10.206.35.234 notec.silence.com
192.168.0.17 master.silence.com
192.168.0.18 nodea.silence.com
192.168.0.19 nodeb.silence.com
cd /etc/yum.repos.d/
scp kubernetes.repo docker-ce.repo note.silence.com:/etc/yum.repos.d/
scp kubernetes.repo docker-ce.repo notec.silence.com:/etc/yum.repos.d/
scp kubernetes.repo docker-ce.repo nodea.silence.com:/etc/yum.repos.d/
- 安装docker-ce:
a) 安装：
yum install docker-ce -y
yum install kubelet kubeadm kubectl -y
b) 报错提示：
c) 解决办法：
i. 手动下载key;
cd
wget https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
rpm --import rpm-package-key.gpg
重新安装即可
yum install docker-ce kubelet kubeadm kubectl
- 启动docker并初始化
a) 修改docker环境变量：
vim /usr/lib/systemd/system/docker.service
Environment="HTTPS_PROXY=http://www.ik8s.io:10080"
Environment="NO_PROXY=127.0.0.0/8,172.172.20.0.0/16"
重新加载：
systemctl daemon-reload
启动docker：
systemctl start docker
查看配置是否生效：
docker info
查看Iptables值是否为1
cat /proc/sys/net/bridge/bridge-nf-call-ip6tables
cat /proc/sys/net/bridge/bridge-nf-call-iptables
如果值为0：
echo "1" >/proc/sys/net/bridge/bridge-nf-call-iptables
echo "1" >/proc/sys/net/bridge/bridge-nf-call-ip6tables - 启动kebulet并初始化
a) 启动服务：
systemctl start kubelet
b) 查看状态：
systemctl status kubelet
c) 查看报错日志：
tail /var/log/messages
d) 关闭Kubelet并开起开机自启：
systemctl stop kubelet
systemctl enable kubelet
systemctl enable docker
e) 初始化前修改配置文件：
vim /etc/sysconfig/kubelet
KUBELET_EXTRA_ARGS="--fail-swap-on=false" - 初始化：
安装：
kubeadm init --kubernetes-version=v1.17.4 --pod-network-cidr=10.244.0.0/16 --service-cidr=10.96.0.0/12 --ignore-preflight-errors=Swap
报错提示：
[ERROR FileAvailable--etc-kubernetes-manifests-etcd.yaml]: /etc/kubernetes/manifests/etcd.yaml already exists
[ERROR Port-10250]: Port 10250 is in use
[ERROR Port-2379]: Port 2379 is in use
[ERROR Port-2380]: Port 2380 is in use
[ERROR DirAvailable--var-lib-etcd]: /var/lib/etcd is not empty
[preflight] If you know what you are doing, you can make a check non-fatal with--ignore-preflight-errors=...
To see the stack trace of this error execute with --v=5 or higher
由于docker和kubelet处于开启状态 - 安装成功：
a) 查看主节点是下载镜像：
docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
k8s.gcr.io/kube-proxy v1.17.4 6dec7cfde1e5 7 days ago 116MB
k8s.gcr.io/kube-apiserver v1.17.4 2e1ba57fe95a 7 days ago 171MB
k8s.gcr.io/kube-controller-manager v1.17.4 7f997fcf3e94 7 days ago 161MB
k8s.gcr.io/kube-scheduler v1.17.4 5db16c1c7aff 7 days ago 94.4MB
k8s.gcr.io/coredns 1.6.5 70f311871ae1 4 months ago 41.6MB
k8s.gcr.io/etcd 3.4.3-0 303ce5db0e90 4 months ago 288MB
k8s.gcr.io/pause 3.1 da86e6ba6ca1 2 years ago 742kB
- 配置master:
a) 查看6443端口是否启用：
ss -tnl
b) 保存master安装相关信息：
kubeadm join 10.206.35.233:6443 --token 0dpyop.njwb331qxuds4vc6 \
--discovery-token-ca-cert-hash sha256:178fa0f54332c1be95d00c003bfa5943b751c2c208cdca2da8d162a1dc595eb4
c) 创建隐藏目录：
mkdir -p $HOME/.kube
d) 复制文件：
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
e) 修改权限：
f) chown $(id -u):$(id -g) $HOME/.kube/config
g) 查看集群组件状态信息：
kubectl get cs/kubectl get componentstatus
h) 查看节点信息：
kubectl get nodes
[root@noteb ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
noteb.silence.com NotReady master 112m v1.17.4
i) 部署网络插件flannel：
i. 安装信息查看：https://github.com/coreos/flannel
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
查看集群状态：变成ready
kubectl get nodes
NAME STATUS ROLES AGE VERSION
noteb.silence.com Ready master 3h28m v1.17.4
ii. 获取所有pods
kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-6955765f44-2gkcf 1/1 Running 0 3h23m
coredns-6955765f44-gl8nx 1/1 Running 0 3h23m
etcd-noteb.silence.com 1/1 Running 0 3h28m
kube-apiserver-noteb.silence.com 1/1 Running 0 3h28m
kube-controller-manager-noteb.silence.com 1/1 Running 1 3h29m
kube-flannel-ds-amd64-wsqd5 1/1 Running 0 21m
kube-proxy-pk6ll 1/1 Running 0 3h23m
kube-scheduler-noteb.silence.com 1/1 Running 1 3h28m
iii. 查看所有名称空间：
kubectl get ns
NAME STATUS AGE
default Active 3h31m
kube-node-lease Active 3h31m
kube-public Active 3h31m
kube-system Active 3h31m - 设置note节点
a) 查看yum源是否存在：
ls /etc/yum.repos.d/
CentOS-Base.repo CentOS-Debuginfo.repo CentOS-Media.repo CentOS-Vault.repo kubernetes.repo
CentOS-CR.repo CentOS-fasttrack.repo CentOS-Sources.repo docker-ce.repo
b) 安装必要的一些系统工具
yum install -y yum-utils device-mapper-persistent-data lvm2
c) 添加软件源信息
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
d) 更新并安装Docker-CE
yum makecache fast
yum install docker-ce -y
e) 配置k8s的yum仓库：
cat </etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
f) 生产yum源：
更新缓存
yum makecache
g)
h) 安装：
yum install kubelet kubeadm -y
i) 修改docker.service文件及Kubenet文件
vim /usr/lib/systemd/system/docker.service
Environment="HTTPS_PROXY=http://www.ik8s.io:10080"
Environment="NO_PROXY=127.0.0.0/8,172.172.20.0.0/16"
vim /etc/sysconfig/kubelet
KUBELET_EXTRA_ARGS="--fail-swap-on=false"
j) 或者从master上复制：
scp /usr/lib/systemd/system/docker.service note.silence.com:/usr/lib/systemd/system/docker.service
scp /usr/lib/systemd/system/docker.service notec.silence.com:/usr/lib/systemd/system/docker.service
scp /usr/lib/systemd/system/docker.service nodea.silence.com:/usr/lib/systemd/system/docker.service
复制kubelet文件
scp /etc/sysconfig/kubelet note.silence.com:/etc/sysconfig/
scp /etc/sysconfig/kubelet notec.silence.com:/etc/sysconfig/
scp /etc/sysconfig/kubelet nodea.silence.com:/etc/sysconfig/
复制kubelet源
scp kubernetes.repo docker-ce.repo nodea.silence.com:/etc/yum.repos.d/
k) 启动docker
systemctl start docker
l) 设置Node上的docker，kubelet开机自启
systemctl enable docker
systemctl enable kubelet
m) 查看环境变量是否启用：
docker info
HTTPS Proxy: http://www.ik8s.io:10080
No Proxy: 127.0.0.0/8,172.172.20.0.0/16
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
如没生效则重启docker
systemctl restart docker
n) 将节点加入集群：
master上运行：
kubeadm token create --print-join-command
得到以下命令：
kubeadm join 10.206.35.233:6443 --token 0dpyop.njwb331qxuds4vc6 --discovery-token-ca-cert-hash sha256:178fa0f54332c1be95d00c003bfa5943b751c2c208cdca2da8d162a1dc595eb4 --ignore-preflight-errors=Swap
kubeadm join 192.168.1.17:6443 --token rczn6r.gkbhtjq6gc7p1efv --discovery-token-ca-cert-hash sha256:97c086ac64a474241c8d09765186396fb3b60e155db84b6235c3738920e3b43c --ignore-preflight-errors=Swap
提示成功如下：
This node has joined the cluster:
- Certificate signing request was sent to apiserver and a response was received.
- The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
o) 查看节点上的镜像：
docker image ls
p) master上是看节点是否加入及相关信息
kubectl get nodes
NAME STATUS ROLES AGE VERSION
note.silence.com NotReady
noteb.silence.com Ready master 4h14m v1.17.4
notec.silence.com NotReady
kubectl get pods -n kube-system -o wide
q) 如果添加节点后显示Noteready:
master上运行：
kubectl describe node note.silence.com
查看：event事件
Events:
查看节点污点：Taints:
Taints: node.kubernetes.io/not-ready:NoExecute
node.kubernetes.io/not-ready:NoSchedule
解决办法，将master上的cni文件复制到各节点中：
scp -r /etc/cni note.silence.com:/etc/cni
scp -r /etc/cni notec.silence.com:/etc/cni
scp -r /etc/cni nodea.silence.com:/etc/cni
各节点上重启Kubelet
systemctl restart kubelet
- 拉镜像到本地：
Master上查看各节点信息
for ima in kube-proxy:v1.12.0 pause:3.1;do docker pull registry.cn-shenzhen.aliyuncs.com/lurenjia/$ima && docker tag registry.cn-shenzhen.aliyuncs.com/lurenjia/$imagcr.io/$ima && docker rmi -f registry.cn-shenzhen.aliyuncs.com/lurenjia/$ima ;done
Master上查看各节点信息
kubectl get pods -n kube-system
查看各节点网卡信息：
ip addr
- 添加节点报错信息
[root@nodea ~]# kubeadm join 10.206.35.233:6443 --token 0dpyop.njwb331qxuds4vc6 --discovery-token-ca-cert-hash sha256:178fa0f54332c1be95d00c003bfa5943b751c2c208cdca2da8d162a1dc595eb4 --ignore-preflight-errors=Swap
W0321 19:54:04.578534 12187 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
[preflight] Running pre-flight checks
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING Hostname]: hostname "nodea.silence.com" could not be reached
[WARNING Hostname]: hostname "nodea.silence.com": lookup nodea.silence.com on 8.8.8.8:53: no such host
error execution phase preflight: couldn't validate the identity of the API Server: abort connecting to API servers after timeout of 5m0s
To see the stack trace of this error execute with --v=5 or higher
a) 解决办法：
重新创建tocken:
kubeadm token create
kubeadm token list
fvglbc.28sfr04ehmg85y2i
178fa0f54332c1be95d00c003bfa5943b751c2c208cdca2da8d162a1dc595eb4
将以下tocken更换：
kubeadm join 10.206.35.233:6443 --token fvglbc.28sfr04ehmg85y2i --discovery-token-ca-cert-hash sha256:178fa0f54332c1be95d00c003bfa5943b751c2c208cdca2da8d162a1dc595eb4 --ignore-preflight-errors=Swap