kubeadm部署1.20.0版本kubernetes
一、环境准备
1.虚拟机环境
| 角色 | 主机名 | IP地址 | 服务组件 |
|---|---|---|---|
| master | master-01 | 192.168.43.100 | comtroller-manager，etcd，scheduler，kube-apiserver |
| node | node-01 | 192.168.43.101 | kubelet，kube-proxy |
| node | node-02 | 192.168.43.102 | kubelet，kube-proxy |
2.配置环境
（1）系统版本
root@ops:/home/ops# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.5 LTS
Release: 18.04
Codename: bionic
（2）修改设置
-
关闭系统交换内存
root@ops:/home/ops# swapoff -a root@ops:/home/ops# free -m total used free shared buff/cache available Mem: 3921 177 3530 1 213 3521 Swap: 0 0 0 root@ops:/home/ops# vi /etc/fstab # -
修改系统参数
root@ops:/home/ops# modprobe overlay root@ops:/home/ops# modprobe br_netfilter # 设置必需的 sysctl 参数，这些参数在重新启动后仍然存在。 root@ops:/home/ops# cat <net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 net.bridge.bridge-nf-call-ip6tables = 1 EOF root@ops:/home/ops# sysctl --system -
设置主机名和host文件
root@ops:/home/ops# cat /etc/hosts 127.0.0.1 localhost 127.0.1.1 ops 192.168.43.100 master-01 192.168.43.101 node-01 192.168.43.102 node-02 root@ops:/home/ops# hostnamectl set-hostname --static master-01 root@ops:/home/ops# exit -
关闭防火墙
root@master-01:~# systemctl stop ufw root@master-01:~# systemctl disable ufw root@master-01:~# ufw disable Firewall stopped and disabled on system startup root@master-01:~# ufw status Status: inactive -
修改时钟和时区
root@master-01:~# ln -snf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime root@master-01:~# bash -c "echo 'Asia/Shanghai' > /etc/timezone" # 安装时间同步软件chrony root@master-01:~# apt install chrony -y # 检查时间同步服务 root@master-01:~# chronyc activity 200 OK 8 sources online 0 sources offline 0 sources doing burst (return to online) 0 sources doing burst (return to offline) 0 sources with unknown address # 检查 Chrony 是否实际同步 root@master-01:~# chronyc tracking Reference ID : CB6B0658 (203.107.6.88) Stratum : 3 Ref time (UTC) : Sun Jul 04 04:51:32 2021 System time : 0.000497580 seconds fast of NTP time Last offset : +0.000895214 seconds RMS offset : 0.000895214 seconds Frequency : 26.221 ppm slow Residual freq : +0.774 ppm Skew : 35.605 ppm Root delay : 0.030476805 seconds Root dispersion : 0.001577590 seconds Update interval : 64.1 seconds Leap status : Normal root@ops:/home/ops# chronyc sources -v 210 Number of sources = 8 .-- Source mode '^' = server, '=' = peer, '#' = local clock. / .- Source state '*' = current synced, '+' = combined , '-' = not combined, | / '?' = unreachable, 'x' = time may be in error, '~' = time too variable. || .- xxxx [ yyyy ] +/- zzzz || Reachability register (octal) -. | xxxx = adjusted offset, || Log2(Polling interval) --. | | yyyy = measured offset, || \ | | zzzz = estimated error. || | | \ MS Name/IP address Stratum Poll Reach LastRx Last sample =============================================================================== ^- ntp.ubuntu.com 2 6 377 25 -2043us[-2043us] +/- 168ms ^- ntp.ubuntu.com 2 6 377 24 +44ms[ +44ms] +/- 148ms ^- golem.canonical.com 2 6 177 21 +49ms[ +49ms] +/- 144ms
3.安装docker-ce
注意docker的安装，可直接参考清华源（https://mirrors.tuna.tsinghua.edu.cn/help/docker-ce/）、阿里云源（https://developer.aliyun.com/mirror/docker-ce?spm=a2c6h.13651102.0.0.3e221b11pq7nqt）等。
此处以清华源为例
- 安装
# 如果过去安装过 docker，先删掉: root@master-01:~# apt-get remove docker docker-engine docker.io # 安装依赖: root@master-01:~# apt-get -y install apt-transport-https ca-certificates curl gnupg2 software-properties-common # 信任 Docker 的 GPG 公钥: root@master-01:~# curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - # 对于amd64 架构的计算机，添加软件仓库: root@master-01:~# add-apt-repository \ "deb [arch=amd64] https://mirrors.tuna.tsinghua.edu.cn/docker-ce/linux/ubuntu \ $(lsb_release -cs) \ stable" # 安装 root@master-01:~# apt-get update # apt-get -y install docker-ce # 设置镜像加速和驱动 root@master-01:~# cat </etc/docker/daemon.json { "registry-mirrors": ["https://docker.mirrors.ustc.edu.cn"], "exec-opts": ["native.cgroupdriver=systemd"] } EOF root@master-01:~# systemctl daemon-reload root@master-01:~# systemctl restart docker.service
4.安装kubelet、kubectl、kubeadm
注意：k8s软件的安装也可以参考阿里云源（https://developer.aliyun.com/mirror/kubernetes?spm=a2c6h.13651102.0.0.3e221b11pq7nqt）和清华源（https://mirrors.tuna.tsinghua.edu.cn/help/kubernetes/）等
-
配置k8s国内源
以阿里云源为例：
root@master-01:~# apt-get update && apt-get install -y apt-transport-https root@master-01:~# curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add - root@master-01:~# cat </etc/apt/sources.list.d/kubernetes.list deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main EOF root@master-01:~# apt-get update root@master-01:~# apt-get install -y kubelet kubeadm kubectl注意：在安装kubelet、kubeadm、kubectl等软件时可指定版本，如kubelet=1.20.0-00 kubeadm=1.20.0-00 kubectl=1.20.0-00
## 查看可安装的k8s版本 root@master-01:~# apt list kubeadm -a Listing... Done kubeadm/kubernetes-xenial 1.24.0-00 amd64 kubeadm/kubernetes-xenial 1.23.6-00 amd64 kubeadm/kubernetes-xenial 1.23.5-00 amd64 kubeadm/kubernetes-xenial 1.23.4-00 amd64 ··· kubeadm/kubernetes-xenial 1.22.4-00 amd64 kubeadm/kubernetes-xenial 1.22.3-00 amd64 kubeadm/kubernetes-xenial 1.22.2-00 amd64 kubeadm/kubernetes-xenial 1.22.1-00 amd64 kubeadm/kubernetes-xenial 1.22.0-00 amd64 kubeadm/kubernetes-xenial 1.21.12-00 amd64 ··· kubeadm/kubernetes-xenial 1.21.0-00 amd64 kubeadm/kubernetes-xenial 1.20.15-00 amd64 ··· kubeadm/kubernetes-xenial 1.6.3-00 amd64 kubeadm/kubernetes-xenial 1.6.2-00 amd64 kubeadm/kubernetes-xenial 1.6.1-00 amd64 kubeadm/kubernetes-xenial 1.5.7-00 amd64 ## 安装指定版本的k8s软件 root@master-01:~# apt-get install -y kubelet=1.20.0-00 kubeadm=1.20.0-00 kubectl=1.20.0-00 Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: conntrack cri-tools kubernetes-cni socat The following NEW packages will be installed: conntrack cri-tools kubeadm kubectl kubelet kubernetes-cni socat 0 upgraded, 7 newly installed, 0 to remove and 57 not upgraded. Need to get 75.1 MB of archives. After this operation, 311 MB of additional disk space will be used. Get:1 https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 cri-tools amd64 1.23.0-00 [15.3 MB] Get:2 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 conntrack amd64 1:1.4.4+snapshot20161117-6ubuntu2 [30.6 kB] Get:3 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] Get:4 https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 kubernetes-cni amd64 0.8.7-00 [25.0 MB] Get:5 https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 kubelet amd64 1.20.0-00 [18.8 MB] Get:6 https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 kubectl amd64 1.20.0-00 [7942 kB] Get:7 https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 kubeadm amd64 1.20.0-00 [7707 kB] Fetched 75.1 MB in 24s (3171 kB/s) Selecting previously unselected package conntrack. (Reading database ... 67508 files and directories currently installed.) Preparing to unpack .../0-conntrack_1%3a1.4.4+snapshot20161117-6ubuntu2_amd64.deb ... Unpacking conntrack (1:1.4.4+snapshot20161117-6ubuntu2) ... Selecting previously unselected package cri-tools. Preparing to unpack .../1-cri-tools_1.23.0-00_amd64.deb ... Unpacking cri-tools (1.23.0-00) ... Selecting previously unselected package kubernetes-cni. Preparing to unpack .../2-kubernetes-cni_0.8.7-00_amd64.deb ... Unpacking kubernetes-cni (0.8.7-00) ... Selecting previously unselected package socat. Preparing to unpack .../3-socat_1.7.3.2-2ubuntu2_amd64.deb ... Unpacking socat (1.7.3.2-2ubuntu2) ... Selecting previously unselected package kubelet. Preparing to unpack .../4-kubelet_1.20.0-00_amd64.deb ... Unpacking kubelet (1.20.0-00) ... Selecting previously unselected package kubectl. Preparing to unpack .../5-kubectl_1.20.0-00_amd64.deb ... Unpacking kubectl (1.20.0-00) ... Selecting previously unselected package kubeadm. Preparing to unpack .../6-kubeadm_1.20.0-00_amd64.deb ... Unpacking kubeadm (1.20.0-00) ... Setting up conntrack (1:1.4.4+snapshot20161117-6ubuntu2) ... Setting up kubernetes-cni (0.8.7-00) ... Setting up cri-tools (1.23.0-00) ... Setting up socat (1.7.3.2-2ubuntu2) ... Setting up kubelet (1.20.0-00) ... Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service → /lib/systemd/system/kubelet.service. Setting up kubectl (1.20.0-00) ... Setting up kubeadm (1.20.0-00) ... Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
二、部署（仅master节点）
1.初始化master节点
root@master-01:~# kubeadm init \
--apiserver-advertise-address=192.168.43.100 \
--kubernetes-version=v1.20.0 \
--image-repository registry.aliyuncs.com/google_containers \
--pod-network-cidr=10.244.0.0/16 \
--service-cidr=10.96.0.0/12 --ignore-preflight-errors=Swap
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.16. Latest validated version: 19.03
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local master-01] and IPs [10.96.0.1 192.168.43.100]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost master-01] and IPs [192.168.43.100 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost master-01] and IPs [192.168.43.100 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 22.005887 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.20" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node master-01 as control-plane by adding the labels "node-role.kubernetes.io/master=''" and "node-role.kubernetes.io/control-plane='' (deprecated)"
[mark-control-plane] Marking the node master-01 as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: k3jj1c.okb3o8vzl90cb57a
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.43.100:6443 --token k3jj1c.okb3o8vzl90cb57a \
--discovery-token-ca-cert-hash sha256:e0159d7cac73cd8201f7e1ce920161f1b990d44f22a8d39825c3cfc00b27bca3
# 如果在内网安装，可使用pull提前下载好镜像，公网则不必如此
# kubeadm config images pull
# 授权用户使用kubectl命令管理集群
ops@master-01:~# mkdir -p $HOME/.kube
root@master-01:~# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
root@master-01:~# chown $(id -u):$(id -g) $HOME/.kube/config
## 永久生效
ops@master-01:~# export KUBECONFIG=$HOME/.kube/config
ops@master-01:~# echo "export KUBECONFIG=$HOME/.kube/config" >> $HOME/.bashrc
ops@master-01:~# source $HOME/.bashrc
参数解释：
-apiserver-advertise-address：设置Master节点API Server的监听地址-kubernetes-version：指定Kubernetes版本-image-repository：由于kubeadm默认是从官网k8s.grc.io下载所需镜像，国内无法访问，所以这里通过–image-repository指定为阿里云镜像仓库地址-pod-network-cidr：指定pod网络段-service-cidr：指定service网络段-ignore-preflight-errors=Swap：忽略swap报错信息
检查集群的健康性：
# 集群健康检查，查看node节点和集群状态
ops@master-01:~$ kubectl get node
NAME STATUS ROLES AGE VERSION
master01 NotReady control-plane,master 8m5s v1.20.0
## 查看集群状态
root@master-01:~# kubectl get cs
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME STATUS MESSAGE ERROR
scheduler Unhealthy Get "http://127.0.0.1:10251/healthz": dial tcp 127.0.0.1:10251: connect: connection refused
controller-manager Unhealthy Get "http://127.0.0.1:10252/healthz": dial tcp 127.0.0.1:10252: connect: connection refused
etcd-0 Healthy {"health":"true"}
## 出现这种情况是kube-controller-manager.yaml和kube-scheduler.yaml文件中均设置的默认端口是0，在文件中注释掉就可以了。(每台master节点都要执行操作)，将下面两个文件中的- --port=0注释掉即可
root@master-01:~# vim /etc/kubernetes/manifests/kube-scheduler.yaml
root@master-01:~# vim /etc/kubernetes/manifests/kube-controller-manager.yaml
root@master-01:~# kubectl get cs
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME STATUS MESSAGE ERROR
controller-manager Healthy ok
scheduler Healthy ok
etcd-0 Healthy {"health":"true"}
## 查看kube-system命名空间下的Pod状态，其中coredns由于网络插件未部署，因此一直是pending状态
root@master-01:~# kubectl get pods -n kube-system -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
coredns-7f89b7bc75-9hxk4 0/1 Pending 0 7m <none> <none> <none> <none>
coredns-7f89b7bc75-p2rjn 0/1 Pending 0 7m <none> <none> <none> <none>
etcd-master-01 1/1 Running 0 7m 192.168.43.100 master-01 <none> <none>
kube-apiserver-master-01 1/1 Running 0 7m 192.168.43.100 master-01 <none> <none>
kube-controller-manager-master-01 1/1 Running 0 7m 192.168.43.100 master-01 <none> <none>
kube-proxy-7m2ct 1/1 Running 0 7m 192.168.43.100 master-01 <none> <none>
kube-scheduler-master-01 1/1 Running 0 7m 192.168.43.100 master-01 <none> <none>
2.加入node节点
在每个node节点上执行
root@node-01:~# kubeadm join 192.168.43.100:6443 --token k3jj1c.okb3o8vzl90cb57a \
--discovery-token-ca-cert-hash sha256:e0159d7cac73cd8201f7e1ce920161f1b990d44f22a8d39825c3cfc00b27bca3
[preflight] Running pre-flight checks
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.16. Latest validated version: 19.03
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
# 拷贝master上的配置文件到所有node节点上，在master上执行
root@master-01:~# scp -P 10022 /etc/kubernetes/admin.conf [email protected]:/home/ops
root@master-01:~# scp -P 10022 /etc/kubernetes/admin.conf [email protected]:/home/ops
# 在node上执行
root@node-01:~# mkdir -p $HOME/.kube
root@node-01:~# cp -i /home/ops/admin.conf $HOME/.kube/config
root@node-01:~# chown $(id -u):$(id -g) $HOME/.kube/config
## 永久生效
ops@master-01:~# export KUBECONFIG=$HOME/.kube/config
ops@master-01:~# echo "export KUBECONFIG=$HOME/.kube/config" >> $HOME/.bashrc
ops@master-01:~# source $HOME/.bashrc
# 可查看node节点信息，由于网络插件未部署，因此status均为NotReady
root@node-01:~# kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
master-01 NotReady control-plane,master 21m v1.20.0 192.168.43.100 <none> Ubuntu 18.04.5 LTS 4.15.0-177-generic docker://20.10.16
node-01 NotReady <none> 3m12s v1.20.0 192.168.43.101 <none> Ubuntu 18.04.5 LTS 4.15.0-177-generic docker://20.10.16
node-02 NotReady <none> 2m58s v1.20.0 192.168.43.102 <none> Ubuntu 18.04.5 LTS 4.15.0-177-generic docker://20.10.16
## 将节点都加入集群后，再次查看kube-system空间的pod
root@master-01:~# kubectl get pods -n kube-system -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
coredns-7f89b7bc75-9hxk4 0/1 Pending 0 27m <none> <none> <none> <none>
coredns-7f89b7bc75-p2rjn 0/1 Pending 0 27m <none> <none> <none> <none>
etcd-master-01 1/1 Running 0 27m 192.168.43.100 master-01 <none> <none>
kube-apiserver-master-01 1/1 Running 0 27m 192.168.43.100 master-01 <none> <none>
kube-controller-manager-master-01 1/1 Running 0 17m 192.168.43.100 master-01 <none> <none>
kube-proxy-7m2ct 1/1 Running 0 27m 192.168.43.100 master-01 <none> <none>
kube-proxy-94p7t 1/1 Running 0 8m56s 192.168.43.101 node-01 <none> <none>
kube-proxy-v8c4s 1/1 Running 0 8m42s 192.168.43.102 node-02 <none> <none>
kube-scheduler-master-01 1/1 Running 0 17m 192.168.43.100 master-01 <none> <none>
默认token有效期为24小时，当过期之后，该token就不可用了。这时就需要重新创建token，可以直接使用命令快捷生成：
root@master-01:~# kubeadm token create --print-join-command
3.命令补全
所有节点上执行
root@master-01:~# apt install -y bash-completion
root@master-01:~# source /usr/share/bash-completion/bash_completion
root@master-01:~# source <(kubectl completion bash)
root@master-01:~# echo "source <(kubectl completion bash)" >> ~/.bashrc
4.部署网络插件（仅在master上执行）
此处以calico网络插件
在未部署网络插件时，kubelet的报错信息如下：
## 此时查看kubelet的日志如下：
root@master-01:~# journalctl -xe
May 22 11:30:50 master-01 kubelet[10648]: E0522 11:30:50.623972 10648 kubelet.go:2160] Container runtime network not ready:
May 22 11:30:55 master-01 kubelet[10648]: W0522 11:30:55.049449 10648 cni.go:239] Unable to update cni config: no networks
May 22 11:30:55 master-01 kubelet[10648]: E0522 11:30:55.638376 10648 kubelet.go:2160] Container runtime network not ready:
May 22 11:31:00 master-01 kubelet[10648]: W0522 11:31:00.050922 10648 cni.go:239] Unable to update cni config: no networks
May 22 11:31:00 master-01 kubelet[10648]: E0522 11:31:00.659415 10648 kubelet.go:2160] Container runtime network not ready:
May 22 11:31:05 master-01 kubelet[10648]: W0522 11:31:05.052126 10648 cni.go:239] Unable to update cni config: no networks
May 22 11:31:05 master-01 kubelet[10648]: E0522 11:31:05.671484 10648 kubelet.go:2160] Container runtime network not ready:
root@master-01:~# tail -f /var/log/syslog
May 22 03:33:16 ops kubelet[10648]: E0522 11:33:16.274186 10648 kubelet.go:2160] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
May 22 03:33:20 ops kubelet[10648]: W0522 11:33:20.074697 10648 cni.go:239] Unable to update cni config: no networks found in /etc/cni/net.d
···
-
上传插件
calico.yaml默认（跟k8s版本保持一致）下载地址为：wget https://docs.projectcalico.org/manifests/calico.yaml，为避免安装插件时产生错误，我已将需要使用的calico.yaml和镜像上传至gitee仓库（上传不了镜像，镜像在docker仓库中）和docker仓库，也可以使用阿里云盘下载
git仓库地址：https://gitee.com/qingdalf/kubernetes.git
阿里云盘地址：https://www.aliyundrive.com/s/i1LNTyyXTUVroot@master-01:~# ls calico-image.tar.gz calico.yaml root@master-01:~# tar xzf calico-image.tar.gz root@master-01:~# ls calico-image.tar.gz calico.yaml cni.tar controllers.tar node.tar pod2daemon.tar root@master-01:~# docker load -i cni.tar root@master-01:~# docker load -i node.tar root@master-01:~# docker load -i pod2daemon.tar root@master-01:~# docker load -i controllers.tar -
calico.yaml文件内容
---
# Source: calico/templates/calico-config.yaml
# This ConfigMap is used to configure a self-hosted Calico installation.
kind: ConfigMap
apiVersion: v1
metadata:
name: calico-config
namespace: kube-system
data:
# Typha is disabled.
typha_service_name: "none"
# Configure the backend to use.
calico_backend: "bird"
# Configure the MTU to use for workload interfaces and tunnels.
# - If Wireguard is enabled, set to your network MTU - 60
# - Otherwise, if VXLAN or BPF mode is enabled, set to your network MTU - 50
# - Otherwise, if IPIP is enabled, set to your network MTU - 20
# - Otherwise, if not using any encapsulation, set to your network MTU.
veth_mtu: "1440"
# The CNI network configuration to install on each node. The special
# values in this config will be automatically populated.
cni_network_config: |-
{
"name": "k8s-pod-network",
"cniVersion": "0.3.1",
"plugins": [
{
"type": "calico",
"log_level": "info",
"datastore_type": "kubernetes",
"nodename": "__KUBERNETES_NODE_NAME__",
"mtu": __CNI_MTU__,
"ipam": {
"type": "calico-ipam"
},
"policy": {
"type": "k8s"
},
"kubernetes": {
"kubeconfig": "__KUBECONFIG_FILEPATH__"
}
},
{
"type": "portmap",
"snat": true,
"capabilities": {"portMappings": true}
},
{
"type": "bandwidth",
"capabilities": {"bandwidth": true}
}
]
}
---
# Source: calico/templates/kdd-crds.yaml
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: (devel)
creationTimestamp: null
name: bgpconfigurations.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: BGPConfiguration
listKind: BGPConfigurationList
plural: bgpconfigurations
singular: bgpconfiguration
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
description: BGPConfiguration contains the configuration for any BGP routing.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: BGPConfigurationSpec contains the values of the BGP configuration.
properties:
asNumber:
description: 'ASNumber is the default AS number used by a node. [Default:
64512]'
format: int32
type: integer
logSeverityScreen:
description: 'LogSeverityScreen is the log severity above which logs
are sent to the stdout. [Default: INFO]'
type: string
nodeToNodeMeshEnabled:
description: 'NodeToNodeMeshEnabled sets whether full node to node
BGP mesh is enabled. [Default: true]'
type: boolean
serviceClusterIPs:
description: ServiceClusterIPs are the CIDR blocks from which service
cluster IPs are allocated. If specified, Calico will advertise these
blocks, as well as any cluster IPs within them.
items:
description: ServiceClusterIPBlock represents a single whitelisted
CIDR block for ClusterIPs.
properties:
cidr:
type: string
type: object
type: array
serviceExternalIPs:
description: ServiceExternalIPs are the CIDR blocks for Kubernetes
Service External IPs. Kubernetes Service ExternalIPs will only be
advertised if they are within one of these blocks.
items:
description: ServiceExternalIPBlock represents a single whitelisted
CIDR External IP block.
properties:
cidr:
type: string
type: object
type: array
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: (devel)
creationTimestamp: null
name: bgppeers.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: BGPPeer
listKind: BGPPeerList
plural: bgppeers
singular: bgppeer
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: BGPPeerSpec contains the specification for a BGPPeer resource.
properties:
asNumber:
description: The AS Number of the peer.
format: int32
type: integer
node:
description: The node name identifying the Calico node instance that
is peering with this peer. If this is not set, this represents a
global peer, i.e. a peer that peers with every node in the deployment.
type: string
nodeSelector:
description: Selector for the nodes that should have this peering. When
this is set, the Node field must be empty.
type: string
peerIP:
description: The IP address of the peer.
type: string
peerSelector:
description: Selector for the remote nodes to peer with. When this
is set, the PeerIP and ASNumber fields must be empty. For each
peering between the local node and selected remote nodes, we configure
an IPv4 peering if both ends have NodeBGPSpec.IPv4Address specified,
and an IPv6 peering if both ends have NodeBGPSpec.IPv6Address specified. The
remote AS number comes from the remote node’s NodeBGPSpec.ASNumber,
or the global default if that is not set.
type: string
required:
- asNumber
- peerIP
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: (devel)
creationTimestamp: null
name: blockaffinities.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: BlockAffinity
listKind: BlockAffinityList
plural: blockaffinities
singular: blockaffinity
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: BlockAffinitySpec contains the specification for a BlockAffinity
resource.
properties:
cidr:
type: string
deleted:
description: Deleted indicates that this block affinity is being deleted.
This field is a string for compatibility with older releases that
mistakenly treat this field as a string.
type: string
node:
type: string
state:
type: string
required:
- cidr
- deleted
- node
- state
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: (devel)
creationTimestamp: null
name: clusterinformations.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: ClusterInformation
listKind: ClusterInformationList
plural: clusterinformations
singular: clusterinformation
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
description: ClusterInformation contains the cluster specific information.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: ClusterInformationSpec contains the values of describing
the cluster.
properties:
calicoVersion:
description: CalicoVersion is the version of Calico that the cluster
is running
type: string
clusterGUID:
description: ClusterGUID is the GUID of the cluster
type: string
clusterType:
description: ClusterType describes the type of the cluster
type: string
datastoreReady:
description: DatastoreReady is used during significant datastore migrations
to signal to components such as Felix that it should wait before
accessing the datastore.
type: boolean
variant:
description: Variant declares which variant of Calico should be active.
type: string
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: (devel)
creationTimestamp: null
name: felixconfigurations.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: FelixConfiguration
listKind: FelixConfigurationList
plural: felixconfigurations
singular: felixconfiguration
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
description: Felix Configuration contains the configuration for Felix.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: FelixConfigurationSpec contains the values of the Felix configuration.
properties:
bpfConnectTimeLoadBalancingEnabled:
description: 'BPFConnectTimeLoadBalancingEnabled when in BPF mode,
controls whether Felix installs the connection-time load balancer. The
connect-time load balancer is required for the host to be able to
reach Kubernetes services and it improves the performance of pod-to-service
connections. The only reason to disable it is for debugging purposes. [Default:
true]'
type: boolean
bpfDataIfacePattern:
description: 'BPFDataIfacePattern is a regular expression that controls
which interfaces Felix should attach BPF programs to in order to
catch traffic to/from the network. This needs to match the interfaces
that Calico workload traffic flows over as well as any interfaces
that handle incoming traffic to nodeports and services from outside
the cluster. It should not match the workload interfaces (usually
named cali...). [Default: ^(en.*|eth.*|tunl0$)]'
type: string
bpfDisableUnprivileged:
description: 'BPFDisableUnprivileged, if enabled, Felix sets the kernel.unprivileged_bpf_disabled
sysctl to disable unprivileged use of BPF. This ensures that unprivileged
users cannot access Calico''s BPF maps and cannot insert their own
BPF programs to interfere with Calico''s. [Default: true]'
type: boolean
bpfEnabled:
description: 'BPFEnabled, if enabled Felix will use the BPF dataplane.
[Default: false]'
type: boolean
bpfExternalServiceMode:
description: 'BPFExternalServiceMode in BPF mode, controls how connections
from outside the cluster to services (node ports and cluster IPs)
are forwarded to remote workloads. If set to "Tunnel" then both
request and response traffic is tunneled to the remote node. If
set to "DSR", the request traffic is tunneled but the response traffic
is sent directly from the remote node. In "DSR" mode, the remote
node appears to use the IP of the ingress node; this requires a
permissive L2 network. [Default: Tunnel]'
type: string
bpfKubeProxyEndpointSlicesEnabled:
description: BPFKubeProxyEndpointSlicesEnabled in BPF mode, controls
whether Felix's embedded kube-proxy accepts EndpointSlices or not.
type: boolean
bpfKubeProxyIptablesCleanupEnabled:
description: 'BPFKubeProxyIptablesCleanupEnabled, if enabled in BPF
mode, Felix will proactively clean up the upstream Kubernetes kube-proxy''s
iptables chains. Should only be enabled if kube-proxy is not running. [Default:
true]'
type: boolean
bpfKubeProxyMinSyncPeriod:
description: 'BPFKubeProxyMinSyncPeriod, in BPF mode, controls the
minimum time between updates to the dataplane for Felix''s embedded
kube-proxy. Lower values give reduced set-up latency. Higher values
reduce Felix CPU usage by batching up more work. [Default: 1s]'
type: string
bpfLogLevel:
description: 'BPFLogLevel controls the log level of the BPF programs
when in BPF dataplane mode. One of "Off", "Info", or "Debug". The
logs are emitted to the BPF trace pipe, accessible with the command
`tc exec bpf debug`. [Default: Off].'
type: string
chainInsertMode:
description: 'ChainInsertMode controls whether Felix hooks the kernel’s
top-level iptables chains by inserting a rule at the top of the
chain or by appending a rule at the bottom. insert is the safe default
since it prevents Calico’s rules from being bypassed. If you switch
to append mode, be sure that the other rules in the chains signal
acceptance by falling through to the Calico rules, otherwise the
Calico policy will be bypassed. [Default: insert]'
type: string
dataplaneDriver:
type: string
debugDisableLogDropping:
type: boolean
debugMemoryProfilePath:
type: string
debugSimulateCalcGraphHangAfter:
type: string
debugSimulateDataplaneHangAfter:
type: string
defaultEndpointToHostAction:
description: 'DefaultEndpointToHostAction controls what happens to
traffic that goes from a workload endpoint to the host itself (after
the traffic hits the endpoint egress policy). By default Calico
blocks traffic from workload endpoints to the host itself with an
iptables “DROP” action. If you want to allow some or all traffic
from endpoint to host, set this parameter to RETURN or ACCEPT. Use
RETURN if you have your own rules in the iptables “INPUT” chain;
Calico will insert its rules at the top of that chain, then “RETURN”
packets to the “INPUT” chain once it has completed processing workload
endpoint egress policy. Use ACCEPT to unconditionally accept packets
from workloads after processing workload endpoint egress policy.
[Default: Drop]'
type: string
deviceRouteProtocol:
description: This defines the route protocol added to programmed device
routes, by default this will be RTPROT_BOOT when left blank.
type: integer
deviceRouteSourceAddress:
description: This is the source address to use on programmed device
routes. By default the source address is left blank, leaving the
kernel to choose the source address used.
type: string
disableConntrackInvalidCheck:
type: boolean
endpointReportingDelay:
type: string
endpointReportingEnabled:
type: boolean
externalNodesList:
description: ExternalNodesCIDRList is a list of CIDR's of external-non-calico-nodes
which may source tunnel traffic and have the tunneled traffic be
accepted at calico nodes.
items:
type: string
type: array
failsafeInboundHostPorts:
description: 'FailsafeInboundHostPorts is a comma-delimited list of
UDP/TCP ports that Felix will allow incoming traffic to host endpoints
on irrespective of the security policy. This is useful to avoid
accidentally cutting off a host with incorrect configuration. Each
port should be specified as tcp:-number> or udp:-number>.
For back-compatibility, if the protocol is not specified, it defaults
to “tcp”. To disable all inbound host ports, use the value none.
The default value allows ssh access and DHCP. [Default: tcp:22,
udp:68, tcp:179, tcp:2379, tcp:2380, tcp:6443, tcp:6666, tcp:6667]'
items:
description: ProtoPort is combination of protocol and port, both
must be specified.
properties:
port:
type: integer
protocol:
type: string
required:
- port
- protocol
type: object
type: array
failsafeOutboundHostPorts:
description: 'FailsafeOutboundHostPorts is a comma-delimited list
of UDP/TCP ports that Felix will allow outgoing traffic from host
endpoints to irrespective of the security policy. This is useful
to avoid accidentally cutting off a host with incorrect configuration.
Each port should be specified as tcp:-number> or udp:-number>.
For back-compatibility, if the protocol is not specified, it defaults
to “tcp”. To disable all outbound host ports, use the value none.
The default value opens etcd’s standard ports to ensure that Felix
does not get cut off from etcd as well as allowing DHCP and DNS.
[Default: tcp:179, tcp:2379, tcp:2380, tcp:6443, tcp:6666, tcp:6667,
udp:53, udp:67]'
items:
description: ProtoPort is combination of protocol and port, both
must be specified.
properties:
port:
type: integer
protocol:
type: string
required:
- port
- protocol
type: object
type: array
genericXDPEnabled:
description: 'GenericXDPEnabled enables Generic XDP so network cards
that don''t support XDP offload or driver modes can use XDP. This
is not recommended since it doesn''t provide better performance
than iptables. [Default: false]'
type: boolean
healthEnabled:
type: boolean
healthHost:
type: string
healthPort:
type: integer
interfaceExclude:
description: 'InterfaceExclude is a comma-separated list of interfaces
that Felix should exclude when monitoring for host endpoints. The
default value ensures that Felix ignores Kubernetes'' IPVS dummy
interface, which is used internally by kube-proxy. If you want to
exclude multiple interface names using a single value, the list
supports regular expressions. For regular expressions you must wrap
the value with ''/''. For example having values ''/^kube/,veth1''
will exclude all interfaces that begin with ''kube'' and also the
interface ''veth1''. [Default: kube-ipvs0]'
type: string
interfacePrefix:
description: 'InterfacePrefix is the interface name prefix that identifies
workload endpoints and so distinguishes them from host endpoint
interfaces. Note: in environments other than bare metal, the orchestrators
configure this appropriately. For example our Kubernetes and Docker
integrations set the ‘cali’ value, and our OpenStack integration
sets the ‘tap’ value. [Default: cali]'
type: string
ipipEnabled:
type: boolean
ipipMTU:
description: 'IPIPMTU is the MTU to set on the tunnel device. See
Configuring MTU [Default: 1440]'
type: integer
ipsetsRefreshInterval:
description: 'IpsetsRefreshInterval is the period at which Felix re-checks
all iptables state to ensure that no other process has accidentally
broken Calico’s rules. Set to 0 to disable iptables refresh. [Default:
90s]'
type: string
iptablesBackend:
description: IptablesBackend specifies which backend of iptables will
be used. The default is legacy.
type: string
iptablesFilterAllowAction:
type: string
iptablesLockFilePath:
description: 'IptablesLockFilePath is the location of the iptables
lock file. You may need to change this if the lock file is not in
its standard location (for example if you have mapped it into Felix’s
container at a different path). [Default: /run/xtables.lock]'
type: string
iptablesLockProbeInterval:
description: 'IptablesLockProbeInterval is the time that Felix will
wait between attempts to acquire the iptables lock if it is not
available. Lower values make Felix more responsive when the lock
is contended, but use more CPU. [Default: 50ms]'
type: string
iptablesLockTimeout:
description: 'IptablesLockTimeout is the time that Felix will wait
for the iptables lock, or 0, to disable. To use this feature, Felix
must share the iptables lock file with all other processes that
also take the lock. When running Felix inside a container, this
requires the /run directory of the host to be mounted into the calico/node
or calico/felix container. [Default: 0s disabled]'
type: string
iptablesMangleAllowAction:
type: string
iptablesMarkMask:
description: 'IptablesMarkMask is the mask that Felix selects its
IPTables Mark bits from. Should be a 32 bit hexadecimal number with
at least 8 bits set, none of which clash with any other mark bits
in use on the system. [Default: 0xff000000]'
format: int32
type: integer
iptablesNATOutgoingInterfaceFilter:
type: string
iptablesPostWriteCheckInterval:
description: 'IptablesPostWriteCheckInterval is the period after Felix
has done a write to the dataplane that it schedules an extra read
back in order to check the write was not clobbered by another process.
This should only occur if another application on the system doesn’t
respect the iptables lock. [Default: 1s]'
type: string
iptablesRefreshInterval:
description: 'IptablesRefreshInterval is the period at which Felix
re-checks the IP sets in the dataplane to ensure that no other process
has accidentally broken Calico’s rules. Set to 0 to disable IP sets
refresh. Note: the default for this value is lower than the other
refresh intervals as a workaround for a Linux kernel bug that was
fixed in kernel version 4.11. If you are using v4.11 or greater
you may want to set this to, a higher value to reduce Felix CPU
usage. [Default: 10s]'
type: string
ipv6Support:
type: boolean
kubeNodePortRanges:
description: 'KubeNodePortRanges holds list of port ranges used for
service node ports. Only used if felix detects kube-proxy running
in ipvs mode. Felix uses these ranges to separate host and workload
traffic. [Default: 30000:32767].'
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
logFilePath:
description: 'LogFilePath is the full path to the Felix log. Set to
none to disable file logging. [Default: /var/log/calico/felix.log]'
type: string
logPrefix:
description: 'LogPrefix is the log prefix that Felix uses when rendering
LOG rules. [Default: calico-packet]'
type: string
logSeverityFile:
description: 'LogSeverityFile is the log severity above which logs
are sent to the log file. [Default: Info]'
type: string
logSeverityScreen:
description: 'LogSeverityScreen is the log severity above which logs
are sent to the stdout. [Default: Info]'
type: string
logSeveritySys:
description: 'LogSeveritySys is the log severity above which logs
are sent to the syslog. Set to None for no logging to syslog. [Default:
Info]'
type: string
maxIpsetSize:
type: integer
metadataAddr:
description: 'MetadataAddr is the IP address or domain name of the
server that can answer VM queries for cloud-init metadata. In OpenStack,
this corresponds to the machine running nova-api (or in Ubuntu,
nova-api-metadata). A value of none (case insensitive) means that
Felix should not set up any NAT rule for the metadata path. [Default:
127.0.0.1]'
type: string
metadataPort:
description: 'MetadataPort is the port of the metadata server. This,
combined with global.MetadataAddr (if not ‘None’), is used to set
up a NAT rule, from 169.254.169.254:80 to MetadataAddr:MetadataPort.
In most cases this should not need to be changed [Default: 8775].'
type: integer
natOutgoingAddress:
description: NATOutgoingAddress specifies an address to use when performing
source NAT for traffic in a natOutgoing pool that is leaving the
network. By default the address used is an address on the interface
the traffic is leaving on (ie it uses the iptables MASQUERADE target)
type: string
natPortRange:
anyOf:
- type: integer
- type: string
description: NATPortRange specifies the range of ports that is used
for port mapping when doing outgoing NAT. When unset the default
behavior of the network stack is used.
pattern: ^.*
x-kubernetes-int-or-string: true
netlinkTimeout:
type: string
openstackRegion:
description: 'OpenstackRegion is the name of the region that a particular
Felix belongs to. In a multi-region Calico/OpenStack deployment,
this must be configured somehow for each Felix (here in the datamodel,
or in felix.cfg or the environment on each compute node), and must
match the [calico] openstack_region value configured in neutron.conf
on each node. [Default: Empty]'
type: string
policySyncPathPrefix:
description: 'PolicySyncPathPrefix is used to by Felix to communicate
policy changes to external services, like Application layer policy.
[Default: Empty]'
type: string
prometheusGoMetricsEnabled:
description: 'PrometheusGoMetricsEnabled disables Go runtime metrics
collection, which the Prometheus client does by default, when set
to false. This reduces the number of metrics reported, reducing
Prometheus load. [Default: true]'
type: boolean
prometheusMetricsEnabled:
description: 'PrometheusMetricsEnabled enables the Prometheus metrics
server in Felix if set to true. [Default: false]'
type: boolean
prometheusMetricsHost:
description: 'PrometheusMetricsHost is the host that the Prometheus
metrics server should bind to. [Default: empty]'
type: string
prometheusMetricsPort:
description: 'PrometheusMetricsPort is the TCP port that the Prometheus
metrics server should bind to. [Default: 9091]'
type: integer
prometheusProcessMetricsEnabled:
description: 'PrometheusProcessMetricsEnabled disables process metrics
collection, which the Prometheus client does by default, when set
to false. This reduces the number of metrics reported, reducing
Prometheus load. [Default: true]'
type: boolean
removeExternalRoutes:
description: Whether or not to remove device routes that have not
been programmed by Felix. Disabling this will allow external applications
to also add device routes. This is enabled by default which means
we will remove externally added routes.
type: boolean
reportingInterval:
description: 'ReportingInterval is the interval at which Felix reports
its status into the datastore or 0 to disable. Must be non-zero
in OpenStack deployments. [Default: 30s]'
type: string
reportingTTL:
description: 'ReportingTTL is the time-to-live setting for process-wide
status reports. [Default: 90s]'
type: string
routeRefreshInterval:
description: 'RouterefreshInterval is the period at which Felix re-checks
the routes in the dataplane to ensure that no other process has
accidentally broken Calico’s rules. Set to 0 to disable route refresh.
[Default: 90s]'
type: string
routeSource:
description: 'RouteSource configures where Felix gets its routing
information. - WorkloadIPs: use workload endpoints to construct
routes. - CalicoIPAM: the default - use IPAM data to construct routes.'
type: string
routeTableRange:
description: Calico programs additional Linux route tables for various
purposes. RouteTableRange specifies the indices of the route tables
that Calico should use.
properties:
max:
type: integer
min:
type: integer
required:
- max
- min
type: object
sidecarAccelerationEnabled:
description: 'SidecarAccelerationEnabled enables experimental sidecar
acceleration [Default: false]'
type: boolean
usageReportingEnabled:
description: 'UsageReportingEnabled reports anonymous Calico version
number and cluster size to projectcalico.org. Logs warnings returned
by the usage server. For example, if a significant security vulnerability
has been discovered in the version of Calico being used. [Default:
true]'
type: boolean
usageReportingInitialDelay:
description: 'UsageReportingInitialDelay controls the minimum delay
before Felix makes a report. [Default: 300s]'
type: string
usageReportingInterval:
description: 'UsageReportingInterval controls the interval at which
Felix makes reports. [Default: 86400s]'
type: string
useInternalDataplaneDriver:
type: boolean
vxlanEnabled:
type: boolean
vxlanMTU:
description: 'VXLANMTU is the MTU to set on the tunnel device. See
Configuring MTU [Default: 1440]'
type: integer
vxlanPort:
type: integer
vxlanVNI:
type: integer
wireguardEnabled:
description: 'WireguardEnabled controls whether Wireguard is enabled.
[Default: false]'
type: boolean
wireguardInterfaceName:
description: 'WireguardInterfaceName specifies the name to use for
the Wireguard interface. [Default: wg.calico]'
type: string
wireguardListeningPort:
description: 'WireguardListeningPort controls the listening port used
by Wireguard. [Default: 51820]'
type: integer
wireguardMTU:
description: 'WireguardMTU controls the MTU on the Wireguard interface.
See Configuring MTU [Default: 1420]'
type: integer
wireguardRoutingRulePriority:
description: 'WireguardRoutingRulePriority controls the priority value
to use for the Wireguard routing rule. [Default: 99]'
type: integer
xdpEnabled:
description: 'XDPEnabled enables XDP acceleration for suitable untracked
incoming deny rules. [Default: true]'
type: boolean
xdpRefreshInterval:
description: 'XDPRefreshInterval is the period at which Felix re-checks
all XDP state to ensure that no other process has accidentally broken
Calico''s BPF maps or attached programs. Set to 0 to disable XDP
refresh. [Default: 90s]'
type: string
required:
- bpfLogLevel
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: (devel)
creationTimestamp: null
name: globalnetworkpolicies.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: GlobalNetworkPolicy
listKind: GlobalNetworkPolicyList
plural: globalnetworkpolicies
singular: globalnetworkpolicy
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
properties:
applyOnForward:
description: ApplyOnForward indicates to apply the rules in this policy
on forward traffic.
type: boolean
doNotTrack:
description: DoNotTrack indicates whether packets matched by the rules
in this policy should go through the data plane's connection tracking,
such as Linux conntrack. If True, the rules in this policy are
applied before any data plane connection tracking, and packets allowed
by this policy are marked as not to be tracked.
type: boolean
egress:
description: The ordered set of egress rules. Each rule contains
a set of packet match criteria and a corresponding action to apply.
items:
description: "A Rule encapsulates a set of match criteria and an
action. Both selector-based security Policy and security Profiles
reference rules - separated out as a list of rules for both ingress
and egress packet matching. \n Each positive match criteria has
a negated version, prefixed with ”Not”. All the match criteria
within a rule must be satisfied for a packet to match. A single
rule can contain the positive and negative version of a match
and both must be satisfied for the rule to match."
properties:
action:
type: string
destination:
description: Destination contains the match criteria that apply
to destination entity.
properties:
namespaceSelector:
description: "NamespaceSelector is an optional field that
contains a selector expression. Only traffic that originates
from (or terminates at) endpoints within the selected
namespaces will be matched. When both NamespaceSelector
and Selector are defined on the same rule, then only workload
endpoints that are matched by both selectors will be selected
by the rule. \n For NetworkPolicy, an empty NamespaceSelector
implies that the Selector is limited to selecting only
workload endpoints in the same namespace as the NetworkPolicy.
\n For NetworkPolicy, `global()` NamespaceSelector implies
that the Selector is limited to selecting only GlobalNetworkSet
or HostEndpoint. \n For GlobalNetworkPolicy, an empty
NamespaceSelector implies the Selector applies to workload
endpoints across all namespaces."
type: string
nets:
description: Nets is an optional field that restricts the
rule to only apply to traffic that originates from (or
terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: NotPorts is the negated version of the Ports
field. Since only some protocols have ports, if any ports
are specified it requires the Protocol match in the Rule
to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: NotSelector is the negated version of the Selector
field. See Selector field for subtleties with negated
selectors.
type: string
ports:
description: "Ports is an optional field that restricts
the rule to only apply to traffic that has a source (destination)
port that matches one of these ranges/values. This value
is a list of integers or strings that represent ranges
of ports. \n Since only some protocols have ports, if
any ports are specified it requires the Protocol match
in the Rule to be set to \"TCP\" or \"UDP\"."
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for sample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching the selector will be matched. \n Note that: in
addition to the negated version of the Selector (see NotSelector
below), the selector expression syntax itself supports
negation. The two types of negation are subtly different.
One negates the set of matched endpoints, the other negates
the whole match: \n \tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled \tendpoints
that do not have the label “my_label”. \n \tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled \tendpoints that do have the label “my_label”.
\n The effect is that the latter will accept packets from
non-Calico sources whereas the former is limited to packets
from Calico-controlled endpoints."
type: string
serviceAccounts:
description: ServiceAccounts is an optional field that restricts
the rule to only apply to traffic that originates from
(or terminates at) a pod running as a matching service
account.
properties:
names:
description: Names is an optional field that restricts
the rule to only apply to traffic that originates
from (or terminates at) a pod running as a service
account whose name is in the list.
items:
type: string
type: array
selector:
description: Selector is an optional field that restricts
the rule to only apply to traffic that originates
from (or terminates at) a pod running as a service
account that matches the given label selector. If
both Names and Selector are specified then they are
AND'ed.
type: string
type: object
type: object
http:
description: HTTP contains match criteria that apply to HTTP
requests.
properties:
methods:
description: Methods is an optional field that restricts
the rule to apply only to HTTP requests that use one of
the listed HTTP Methods (e.g. GET, PUT, etc.) Multiple
methods are OR'd together.
items:
type: string
type: array
paths:
description: 'Paths is an optional field that restricts
the rule to apply to HTTP requests that use one of the
listed HTTP Paths. Multiple paths are OR''d together.
e.g: - exact: /foo - prefix: /bar NOTE: Each entry may
ONLY specify either a `exact` or a `prefix` match. The
validator will check for it.'
items:
description: 'HTTPPath specifies an HTTP path to match.
It may be either of the form: exact: -
应用插件
注意：如果在kubeadm init时–pod-network-cidr的网段使用的不是10.244.0.0/16，请将calico.yaml中CALICO_IPV4POOL_CIDR对应的value值修改成初始化时–pod-network-cidr的网段；若一致，则直接应用插件即可。
## 应用插件 root@master-01:~# kubectl apply -f calico.yaml configmap/calico-config created customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created clusterrole.rbac.authorization.k8s.io/calico-kube-controllers created clusterrolebinding.rbac.authorization.k8s.io/calico-kube-controllers created clusterrole.rbac.authorization.k8s.io/calico-node created clusterrolebinding.rbac.authorization.k8s.io/calico-node created daemonset.apps/calico-node created serviceaccount/calico-node created deployment.apps/calico-kube-controllers created serviceaccount/calico-kube-controllers created ## 执行完成后，可在集群node节点上查看集群中的node和pods，状态都恢复正常 root@master-01:~# kubectl get nodes NAME STATUS ROLES AGE VERSION master-01 Ready control-plane,master 35h v1.20.0 node-01 Ready <none> 35h v1.20.0 node-02 Ready <none> 35h v1.20.0 root@master-01:~# kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE calico-kube-controllers-97769f7c7-4bmg8 1/1 Running 1 5m39s calico-node-jkrlp 1/1 Running 1 3m50s calico-node-l9k5k 1/1 Running 0 3m50s calico-node-sf9ht 1/1 Running 0 3m50s coredns-7f89b7bc75-9hxk4 1/1 Running 1 35h coredns-7f89b7bc75-p2rjn 1/1 Running 1 35h etcd-master-01 1/1 Running 1 35h kube-apiserver-master-01 1/1 Running 1 35h kube-controller-manager-master-01 1/1 Running 1 35h kube-proxy-7m2ct 1/1 Running 1 35h kube-proxy-94p7t 1/1 Running 1 35h kube-proxy-v8c4s 1/1 Running 1 35h kube-scheduler-master-01 1/1 Running 1 35h注意：
kube-system命名空间中的calico-kube-controllers、coredns、etcd、kube-apiserver、kube-controller-manager、kube-scheduler只分布在mster节点上，calico-node、kube-proxy均匀分布在集群中的每个节点上，其中etcd、kube-apiserver、kube-controller-manager、kube-scheduler四个静态Pod（不受调度器管理，只存在于特定节点上，yaml文件默认是存放于/etc/kubernetes/manifests/目录下）命名规则是xxx-master的主机名
3.验证
此处通过新建nginx的一个deployment，然后访问pod、svc、node的IP查看网络情况
## 创建deployment
root@master-01:~# kubectl create deployment nginx --image=nginx
deployment.apps/nginx created
## 创建svc
root@master-01:~# kubectl expose deployment nginx --port=80 --type=NodePort
service/nginx exposed
## 查看pod和svc
root@master-01:~# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-6799fc88d8-nmqkv 1/1 Running 0 37s 10.244.190.1 node-01 <none> <none>
root@master-01:~# kubectl get svc -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 36h <none>
nginx NodePort 10.97.193.178 <none> 80:30165/TCP 7s app=nginx
## 通过pod的IP地址访问
root@master-01:~# curl -I 10.244.190.1
HTTP/1.1 200 OK
Server: nginx/1.21.5
Date: Sun, 22 May 2022 04:21:23 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Tue, 28 Dec 2021 15:28:38 GMT
Connection: keep-alive
ETag: "61cb2d26-267"
Accept-Ranges: bytes
## 通过svc的IP地址访问
root@master-01:~# curl -I 10.97.193.178
HTTP/1.1 200 OK
Server: nginx/1.21.5
Date: Sun, 22 May 2022 04:21:36 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Tue, 28 Dec 2021 15:28:38 GMT
Connection: keep-alive
ETag: "61cb2d26-267"
Accept-Ranges: bytes
## 通过集群所有节点的IP地址访问
root@master-01:~# curl -I 192.168.43.100::30165 # 通过master的IP+端口访问
HTTP/1.1 200 OK
Server: nginx/1.21.5
Date: Sun, 22 May 2022 04:21:49 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Tue, 28 Dec 2021 15:28:38 GMT
Connection: keep-alive
ETag: "61cb2d26-267"
Accept-Ranges: bytes
root@master-01:~# curl -I 192.168.43.101:30165 # 通过node-01的IP+端口访问
HTTP/1.1 200 OK
Server: nginx/1.21.5
Date: Sun, 22 May 2022 04:23:09 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Tue, 28 Dec 2021 15:28:38 GMT
Connection: keep-alive
ETag: "61cb2d26-267"
Accept-Ranges: bytes
root@master-01:~# curl -I 192.168.43.102:30165 # 通过node-02的IP+端口访问
HTTP/1.1 200 OK
Server: nginx/1.21.5
Date: Sun, 22 May 2022 04:23:12 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Tue, 28 Dec 2021 15:28:38 GMT
Connection: keep-alive
ETag: "61cb2d26-267"
Accept-Ranges: bytes
通过浏览器访问，结果如下：
5. 部署dashboard插件
root@master-01:~# ls kubernertes-dashboard.yaml
kubernertes-dashboard.yaml
root@master-01:~# kubectl apply -f kubernertes-dashboard.yaml
namespace/kubernetes-dashboard created
serviceaccount/kubernetes-dashboard created
service/kubernetes-dashboard created
secret/kubernetes-dashboard-certs created
secret/kubernetes-dashboard-csrf created
secret/kubernetes-dashboard-key-holder created
configmap/kubernetes-dashboard-settings created
role.rbac.authorization.k8s.io/kubernetes-dashboard created
clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard created
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
deployment.apps/kubernetes-dashboard created
service/dashboard-metrics-scraper created
deployment.apps/dashboard-metrics-scraper created
# 查看创建的pod
root@master-01:~# kubectl get pods -n kubernetes-dashboard
NAME READY STATUS RESTARTS AGE
dashboard-metrics-scraper-7b59f7d4df-8wwlb 1/1 Running 0 59s
kubernetes-dashboard-5dbf55bd9d-n5z7j 1/1 Running 0 59s
## 查看svc
root@master-01:~# kubectl get svc -n kubernetes-dashboard
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
dashboard-metrics-scraper ClusterIP 10.106.233.144 <none> 8000/TCP 4m23s
kubernetes-dashboard NodePort 10.107.126.226 <none> 443:30001/TCP 4m23s
## 此时，集群中的所有node节点，都开启了30001端口，用户可使用任意node的ip地址加30001访问dashboard页面
root@master-01:~# netstat -lntp | grep 30001
tcp 0 0 0.0.0.0:30001 0.0.0.0:* LISTEN 3313/kube-proxy
root@node-01:~# netstat -lntp | grep 30001
tcp 0 0 0.0.0.0:30001 0.0.0.0:* LISTEN 2442/kube-proxy
root@node-02:~# netstat -lntp | grep 30001
tcp 0 0 0.0.0.0:30001 0.0.0.0:* LISTEN 2442/kube-proxy
## 查看pod日志，如果有报错的话，把pod删除掉，重启一个
root@master-01:~# kubetctl -n kubernetes-dashboard describe pods kubernetes-dashboard-5dbf55bd9d-n5z7j
···
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 7m42s default-scheduler Successfully assigned kubernetes-dashboard/kubernetes-dashboard-5dbf55bd9d-n5z7j to node-02
Warning FailedMount 7m40s kubelet MountVolume.SetUp failed for volume "kubernetes-dashboard-certs" : failed to sync secret cache: timed out waiting for the condition
Normal Pulling 7m39s kubelet Pulling image "kubernetesui/dashboard:v2.0.3"
Normal Pulled 6m54s kubelet Successfully pulled image "kubernetesui/dashboard:v2.0.3" in 44.609908262s
Normal Created 6m54s kubelet Created container kubernetes-dashboard
Normal Started 6m53s kubelet Started container kubernetes-dashboard
root@master-01:~# kubetctl -n kubernetes-dashboard delete pods kubernetes-dashboard-5dbf55bd9d-n5z7j
## 日志输出如下就没问题了
root@master-01:~# kubectl -n kubernetes-dashboard describe pods kubernetes-dashboard-5dbf55bd9d-c4j7
···
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 88s default-scheduler Successfully assigned kubernetes-dashboard/kubernetes-dashboard-5dbf55bd9d-c4j7d to node-01
Normal Pulling 86s kubelet Pulling image "kubernetesui/dashboard:v2.0.3"
Normal Pulled 60s kubelet Successfully pulled image "kubernetesui/dashboard:v2.0.3" in 26.000595615s
Normal Created 59s kubelet Created container kubernetes-dashboard
Normal Started 59s kubelet Started container kubernetes-dashboard
通过浏览器访问：https://NodeIP:30001（通过集群中的任意一个node的IP加端口30001都能访问）
创建service account并绑定默认cluster-admin管理员集群角色：
root@master-01:~# kubectl create serviceaccount dashboard-admin -n kube-system
serviceaccount/dashboard-admin created
root@master-01:~# kubectl create clusterrolebinding dashboard-admin --clusterrole=cluster-admin --serviceaccount=kube-system:dashboard-admin
clusterrolebinding.rbac.authorization.k8s.io/dashboard-admin created
root@master-01:~# kubectl describe secrets -n kube-system $(kubectl -n kube-system get secret | awk '/dashboard-admin/{print $1}')
Name: dashboard-admin-token-6xnq2
Namespace: kube-system
Labels: <none>
Annotations: kubernetes.io/service-account.name: dashboard-admin
kubernetes.io/service-account.uid: 033e7b52-01e5-4a09-a3a2-1aaa835af222
Type: kubernetes.io/service-account-token
Data
====
ca.crt: 1066 bytes
namespace: 11 bytes
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IlJfSTM3UzRSTnBjM2ZmQWxTZGEyZ1JzcjVETGVheXo3NGxaRW94bnA5MGMifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4tNnhucTIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiMDMzZTdiNTItMDFlNS00YTA5LWEzYTItMWFhYTgzNWFmMjIyIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9.Q-Ee6HDMZ3LSr7VYq6lqq7LFqm5JtbYKB55CE82hvN7DC6yBDxGI3Qv5XfckKRk-imUl6ULf747i_yqleiOLGGmhDhgp8nw21ZT9DsNWh8eNs28yuumjS7p8LyJCAE-wpeIQQZLx2JgRoAFBIRGFYY1yH-2ccLCh5FLmSMqB60MZSXhMuwHDJPk0tGIhdGfLypjMtACBsOIGyMtGu49q4SENym0SvbEjatXh3KeIR1t0YQnMKJkGHwHxLd0eVfsaj5PeOUSTURuwUAGLd2Sa2NXloFR_6vEvw3YRustPAMLyuI4Ue-qjljWHi8b5N1E5DXWN-_-T08VBcLbH3RQWew
地址：https://nodeip:30001（此处的nodeIP,可以是节点IP也可以是master的IP），在登录界面输入Token值，登录dashboard页面。
dashboard界面如下：
三、Kubectl的使用
- 格式：
kubectl [-n namespace] 操作命令 资源对象 资源对象名称，如kubectl -n kube-system get pods表示查看kube-system命名空间中的所有Pod的信息。
# kubectl --help
kubectl controls the Kubernetes cluster manager.
Find more information at: https://kubernetes.io/docs/reference/kubectl/overview/
Basic Commands (Beginner):
create Create a resource from a file or from stdin.
expose Take a replication controller, service, deployment or pod and expose it as a new Kubernetes Service
run Run a particular image on the cluster
set Set specific features on objects
Basic Commands (Intermediate):
explain Documentation of resources # 资源文档信息
get Display one or many resources # 展示资源
edit Edit a resource on the server # 编辑资源信息
delete Delete resources by filenames, stdin, resources and names, or by resources and label selector # 删除资源
Deploy Commands:
rollout Manage the rollout of a resource
scale Set a new size for a Deployment, ReplicaSet or Replication Controller
autoscale Auto-scale a Deployment, ReplicaSet, or ReplicationController
Cluster Management Commands: # 集群管理命令
certificate Modify certificate resources. # 修改证书认证信息
cluster-info Display cluster info # 显示集群信息
top Display Resource (CPU/Memory/Storage) usage. # 展示集群资源利用率
cordon Mark node as unschedulable # 标记node(计算节点)不可被调度，即不允许在该节点上创建pod
uncordon Mark node as schedulable # 恢复node(计算节点)使用，允许被调度
drain Drain node in preparation for maintenance # 将node(计算节点)上的pod迁移到其他node节点
taint Update the taints on one or more nodes
Troubleshooting and Debugging Commands: # 排错和debug命令
describe Show details of a specific resource or group of resources
logs Print the logs for a container in a pod
attach Attach to a running container
exec Execute a command in a container
port-forward Forward one or more local ports to a pod
proxy Run a proxy to the Kubernetes API server
cp Copy files and directories to and from containers.
auth Inspect authorization
debug Create debugging sessions for troubleshooting workloads and nodes
Advanced Commands: # 高级命令
diff Diff live version against would-be applied version
apply Apply a configuration to a resource by filename or stdin
patch Update field(s) of a resource
replace Replace a resource by filename or stdin
wait Experimental: Wait for a specific condition on one or many resources.
kustomize Build a kustomization target from a directory or a remote url.
Settings Commands: # 设置命令
label Update the labels on a resource
annotate Update the annotations on a resource
completion Output shell completion code for the specified shell (bash or zsh)
Other Commands: # 其他命令
api-resources Print the supported API resources on the server
api-versions Print the supported API versions on the server, in the form of "group/version"
config Modify kubeconfig files # 修改kubeconfig文件
plugin Provides utilities for interacting with plugins.
version Print the client and server version information
Usage: # 基本用法
kubectl [flags] [options]
Use "kubectl --help" for more information about a given command.
Use "kubectl options" for a list of global command-line options (applies to all commands).