K8s和KubeEdge v1.6.1安装及应用部署到DLAP221指南
1. 系统配置
1.1 系统环境
ke-cloud 云端 192.168.31.4 k8s、docker、cloudcore
ke-edge1 边缘端 192.168.2.21 docker、edgecore
为了更能接近实际生产环境，此实验环境采用跨网段
网络结构图：
internet
|
电信拨号router（192.168.31.1）
|---------------|----------------|
PC server(192.168.31.4) router（192.168.31.2/DHCP server(192.168.2/24)
|
DLAP221(动态获取子网ip：192.168.2.21)
2. 安装Docker
在云端和边缘端都安装docker
$sudo apt-get install docker.io
$sudo docker version
Client:
Version: 20.10.2
API version: 1.41
Go version: go1.13.8
Git commit: 20.10.2-0ubuntu1~18.04.2
Built: Tue Mar 30 21:24:16 2021
OS/Arch: linux/amd64
Context: default
Experimental: true
Server:
Engine:
Version: 20.10.2
API version: 1.41 (minimum version 1.12)
Go version: go1.13.8
Git commit: 20.10.2-0ubuntu1~18.04.2
Built: Mon Mar 29 19:27:41 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.3.3-0ubuntu1~18.04.4
GitCommit:
runc:
Version: spec: 1.0.2-dev
GitCommit:
docker-init:
Version: 0.19.0
GitCommit:
$systemctl enable docker.service && systemctl start docker
3. 安装kubernetes
3.1 关闭swap
swap打开的情况下，kubelet无法正常运行，journalctl -xefu kubelet可查看日志；
关闭swap主要是为了性能考虑，kubernetes的想法是将实例紧密包装到尽可能接近100％。 所有的部署应该与CPU /内存限制固定在一起。 所以如果调度程序发送一个pod到一台机器，它不应该使用交换。 设计者不想交换，因为它会减慢速度。
关于swap的讨论1，2
$sudo swapoff -a
$ free -h
total used free shared buff/cache available
Mem: 7.7G 2.3G 3.3G 633M 2.0G 4.5G
Swap: 0B 0B 0B
3.2 安装kubeadm
$ sudo apt update && sudo apt install -y apt-transport-https curl
$ curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add -
/etc/apt/sources.list.d/kubernetes.list中添加aliyun的镜像地址
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
sudo apt update
sudo apt install -y kubelet kubeadm kubectl
安装之后查看版本
$ kubelet --version
Kubernetes v1.20.5
3.3 其他组件介绍
Kubernetes主要由以下几个核心组件组成：
etcd保存了整个集群的状态；
apiserver提供了资源操作的唯一入口，并提供认证、授权、访问控制、API注册和发现等机制；
controller manager负责维护集群的状态，比如故障检测、自动扩展、滚动更新等；
scheduler负责资源的调度，按照预定的调度策略将Pod调度到相应的机器上；
kubelet负责维护容器的生命周期，同时也负责Volume（CVI）和网络（CNI）的管理；
Container runtime负责镜像管理以及Pod和容器的真正运行（CRI）；
kube-proxy负责为Service提供cluster内部的服务发现和负载均衡；
kubeadm init这个命令帮助你启动跟Master相关的组件APIServer、Etcd、Scheduler、Controller-Manager等。
由于官方镜像地址被墙，所以我们需要首先获取所需镜像以及它们的版本。然后从国内镜像站获取。
手动拉取镜像的方法如下（不推荐该方法，推荐使用--image-repository参数）：用命令查看版本当前kubeadm对应的k8s组件镜像版本，如下：
[root@ke-cloud kubeedge]# kubeadm config images list I0201 01:11:35.698583 15417 version.go:251] remote version is much newer: v1.20.2; falling back to: stable-1.17 W0201 01:11:39.538445 15417 validation.go:28] Cannot validate kube-proxy config - no validator is available W0201 01:11:39.538486 15417 validation.go:28] Cannot validate kubelet config - no validator is available k8s.gcr.io/kube-apiserver:v1.17.17 k8s.gcr.io/kube-controller-manager:v1.17.17 k8s.gcr.io/kube-scheduler:v1.17.17 k8s.gcr.io/kube-proxy:v1.17.17 k8s.gcr.io/pause:3.1 k8s.gcr.io/etcd:3.4.3-0 k8s.gcr.io/coredns:1.6.5使用kubeadm config images pull命令拉取上述镜像，如下：
[root@ke-cloud ~]# kubeadm config images pull I0201 01:11:35.188139 6015 version.go:251] remote version is much newer: v1.18.6; falling back to: stable-1.17 I0201 01:11:35.580861 6015 validation.go:28] Cannot validate kube-proxy config - no validator is available I0201 01:11:35.580877 6015 validation.go:28] Cannot validate kubelet config - no validator is available [config/images] Pulled k8s.gcr.io/kube-apiserver:v1.17.17 [config/images] Pulled k8s.gcr.io/kube-controller-manager:v1.17.17 [config/images] Pulled k8s.gcr.io/kube-scheduler:v1.17.17 [config/images] Pulled k8s.gcr.io/kube-proxy:v1.17.17 [config/images] Pulled k8s.gcr.io/pause:3.1 [config/images] Pulled k8s.gcr.io/etcd:3.4.3-0 [config/images] Pulled k8s.gcr.io/coredns:1.6.5查看下载下来的镜像，如下：
[root@ke-cloud ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE k8s.gcr.io/kube-proxy v1.17.17 3ef67d180564 2 weeks ago 117MB k8s.gcr.io/kube-apiserver v1.17.17 38db32e0f351 2 weeks ago 171MB k8s.gcr.io/kube-controller-manager v1.17.17 0ddd96ecb9e5 2 weeks ago 161MB k8s.gcr.io/kube-scheduler v1.17.17 d415ebbf09db 2 weeks ago 94.4MB quay.io/coreos/flannel v0.13.1-rc1 f03a23d55e57 2 months ago 64.6MB k8s.gcr.io/coredns 1.6.5 70f311871ae1 15 months ago 41.6MB k8s.gcr.io/etcd 3.4.3-0 303ce5db0e90 15 months ago 288MB k8s.gcr.io/pause 3.1 da86e6ba6ca1 3 years ago 742kB
以上方法比较繁琐，如果想直接从国内镜像仓库拉取镜像，可以在kubeadm init 时增加参数--image-repository registry.aliyuncs.com/google_containers。执行kubeadm init，需要指明pod网络可以使用的IP地址段，即‘–pod-network-cidr’，如果安装flannel，参数为--pod-network-cidr=10.244.0.0/16，安装calico，参数为‘–pod-network-cidr=192.168.0.0/16’。
3.4. 初始化集群
$sudo kubeadm init --kubernetes-version=v1.20.5 \
--pod-network-cidr=10.244.0.0/16 \
--apiserver-advertise-address=192.168.31.4 \
--image-repository registry.aliyuncs.com/google_containers
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.31.4:6443 --token nvjzj0.dvk4n3hvj38cpi3t \
--discovery-token-ca-cert-hash sha256:86bf19cb4c924ef2f45c6c3d792cca0a4c94e0bfefa7575e216d8ede8ba3b994
进一步配置kubectl
$ rm -rf $HOME/.kube
$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config
查看node节点
$ kubectl get node
NAME STATUS ROLES AGE VERSION
chenx-latitude-3410 NotReady control-plane,master 10m v1.20.5
查看安装情况
$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-7f89b7bc75-hb88t 0/1 Pending 0 32m
kube-system coredns-7f89b7bc75-vh48p 0/1 Pending 0 32m
kube-system etcd-chenx-latitude-3410 1/1 Running 0 33m
kube-system kube-apiserver-chenx-latitude-3410 1/1 Running 0 33m
kube-system kube-controller-manager-chenx-latitude-3410 1/1 Running 0 33m
kube-system kube-proxy-bgnvp 1/1 Running 0 32m
kube-system kube-scheduler-chenx-latitude-3410 1/1 Running 0 33m
发现所有的coredns pod都处于Pending状态，我们还需要安装Pod Network插件， kubeadm only supports Container Network Interface (CNI) based networks (and does not support kubenet).
这里使用flannel网络
3.5 配置网络插件
下载flannel插件的yaml文件
$ cd ~ && mkdir flannel && cd flannel
$ curl -O https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
启动
$kubectl apply -f ~/flannel/kube-flannel.yml
查看
$ kubectl get node
NAME STATUS ROLES AGE VERSION
chenx-latitude-3410 Ready control-plane,master 43m v1.20.5
$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-7f89b7bc75-hb88t 0/1 Running 0 41m
kube-system coredns-7f89b7bc75-vh48p 0/1 Running 0 41m
kube-system etcd-chenx-latitude-3410 1/1 Running 0 41m
kube-system kube-apiserver-chenx-latitude-3410 1/1 Running 0 41m
kube-system kube-controller-manager-chenx-latitude-3410 1/1 Running 0 41m
kube-system kube-flannel-ds-6jvlr 1/1 Running 0 40s
kube-system kube-proxy-bgnvp 1/1 Running 0 41m
kube-system kube-scheduler-chenx-latitude-3410 1/1 Running 0 41m
$kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 45m
所有的pod状态都变为Running
注意：只有网络插件也安装配置完成之后，node才能会显示为ready状态。
到此，说明kubernetes安装完成啦。
设置master节点也可以运行pod(可选)
$ kubectl taint nodes --all node-role.kubernetes.io/master-
node/zml untainted
$ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
zml Ready master 32m v1.16.2 192.0.4.138 <none> Ubuntu 18.04.3 LTS 4.15.0-65-generic docker://19.3.3
3.6 配置kubelet（可选）
在cloud端配置kubelet并非必须，主要是为了验证K8s集群的部署是否正确，也可以在云端搭建Dashboard等应用。
获取Docker的cgroups
$ DOCKER_CGROUPS=$(docker info | grep 'Cgroup' | cut -d' ' -f4)
$ echo $DOCKER_CGROUPS
cgroupfs
配置kubelet的cgroups
$ cat >/etc/sysconfig/kubelet<<EOF
KUBELET_EXTRA_ARGS="--cgroup-driver=$DOCKER_CGROUPS --pod-infra-container-image=k8s.gcr.io/pause:3.1"
EOF
启动kubelet
$ systemctl daemon-reload
$ systemctl enable kubelet && systemctl start kubelet
特别说明：在这里使用systemctl status kubelet会发现报错误信息，这个错误在运行kubeadm init 生成CA证书后会被自动解决，此处可先忽略。
4. KubeEdge在线部署{#1}
4.1 cloud端配置
cloud端负责编译KubeEdge的相关组件与运行cloudcore。
4.1.1 准备工作
创建部署工程目录：
$ mkdir /home/HwHiAiUser/kubeedge
创建子目录：
$ cd /home/HwHiAiUser/kubeedge
$ mkdir src
下载golang
$ wget https://studygolang.com/dl/golang/go1.16.3.linux-amd64.tar.gz
$ sudo tar -C /usr/local -xzf go1.16.3.linux-amd64.tar.gz
配置golang环境
$ vim ~/.bashrc
文件末尾添加：
# golang env
export GOROOT=/usr/local/go
export GOPATH=/home/HwHiAiUser/kubeedge/gopath
export PATH=$PATH:$GOROOT/bin:$GOPATH/bin
$ source ~/.bashrc
$ mkdir -p ~/kubeedge/gopath && cd ~/kubeedge/gopath
$ mkdir -p src pkg bin
下载KubeEdge源码
$ git clone https://github.com/kubeedge/kubeedge $GOPATH/src/github.com/kubeedge/kubeedge
git clone https://github.com/daixiang0/kubeedge $GOPATH/src/github.com/kubeedge/kubeedge
4.1.2 部署cloudcore
编译kubeadm
$ cd $GOPATH/src/github.com/kubeedge/kubeedge
$ make all WHAT=keadm
说明：编译后的二进制文件在./_output/local/bin下，单独编译cloudcore的方式如下：
$ make all WHAT=cloudcore
将编译好的二进制文件 copy 到/usr/local/bin 中
$ sudo cp _output/local/bin/* /usr/local/bin/
创建cloud节点
$ keadm init --advertise-address="192.168.31.4"
Kubernetes version verification passed, KubeEdge installation will start...
kubeedge-v1.6.1-linux-amd64.tar.gz checksum:
checksum_kubeedge-v1.6.1-linux-amd64.tar.gz.txt content:
[Run as service] start to download service file for cloudcore
[Run as service] success to download service file for cloudcore
kubeedge-v1.6.1-linux-amd64/
kubeedge-v1.6.1-linux-amd64/edge/
kubeedge-v1.6.1-linux-amd64/edge/edgecore
kubeedge-v1.6.1-linux-amd64/cloud/
kubeedge-v1.6.1-linux-amd64/cloud/csidriver/
kubeedge-v1.6.1-linux-amd64/cloud/csidriver/csidriver
kubeedge-v1.6.1-linux-amd64/cloud/admission/
kubeedge-v1.6.1-linux-amd64/cloud/admission/admission
kubeedge-v1.6.1-linux-amd64/cloud/cloudcore/
kubeedge-v1.6.1-linux-amd64/cloud/cloudcore/cloudcore
kubeedge-v1.6.1-linux-amd64/version
KubeEdge cloudcore is running, For logs visit: /var/log/kubeedge/cloudcore.log
CloudCore started
4.2 edge端配置
由于我们的edge端的架构是ARM64，所以edgecore，需要交叉编译或者再edge端编译，这里我们采用在edge端编译edgecore
4.2.1 准备工作
创建部署工程目录：
$ mkdir /home/HwHiAiUser/kubeedge
创建子目录：
$ cd /home/HwHiAiUser/kubeedge
$ mkdir src
下载golang
$ wget https://studygolang.com/dl/golang/go1.16.3.linux-arm64.tar.gz
$ sudo tar -C /usr/local -xzf go1.16.3.linux-arm64.tar.gz
配置golang环境
$ vim ~/.bashrc
文件末尾添加：
# golang env
export GOROOT=/usr/local/go
export GOPATH=/home/HwHiAiUser/kubeedge/gopath
export PATH=$PATH:$GOROOT/bin:$GOPATH/bin
$ source ~/.bashrc
$ mkdir -p ~/kubeedge/gopath && cd ~/kubeedge/gopath
$ mkdir -p src
下载KubeEdge源码
$ git clone https://github.com/kubeedge/kubeedge $GOPATH/src/github.com/kubeedge/kubeedge
$ git branch -a
* master
remotes/origin/HEAD -> origin/master
remotes/origin/master
remotes/origin/release-0.3
remotes/origin/release-1.0
remotes/origin/release-1.1
remotes/origin/release-1.2
remotes/origin/release-1.3
remotes/origin/release-1.4
remotes/origin/release-1.5
remotes/origin/release-1.6
$ git checkout -b release-1.6 remotes/origin/release-1.6
编译edgecore
$ cd $GOPATH/src/github.com/kubeedge/kubeedge
$ make all WHAT=keadm
说明：编译后的二进制文件在./_output/local/bin下，单独编译edgecore的方式如下：
$ make all WHAT=edgecore
将编译好的二进制文件 copy 到/usr/local/bin 中
$ sudo cp _output/local/bin/* /usr/local/bin/
4.2.2 从云端获取令牌
在云端运行将返回令牌，该令牌将在加入边缘节点时使用。keadm gettoken
$ keadm gettoken
0fdaccdb675b34b7e4f7b722867d3789d429c94b9ea69cf36f1905a3525ba9c8.eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2MTc5NTM1Mjl9.hqfTEhZPx_H_FDEUSnqhBenDHd6lyppYG2qtP0Kgw90
4.2.3 加入边缘节点
keadm join将安装edgecore和mqtt。它还提供了一个标志，通过它可以设置特定的版本。
$ keadm join --cloudcore-ipport=192.168.31.4:10000 --token=0fdaccdb675b34b7e4f7b722867d3789d429c94b9ea69cf36f1905a3525ba9c8.eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2MTc5NTM1Mjl9.hqfTEhZPx_H_FDEUSnqhBenDHd6lyppYG2qtP0Kgw90
Expected or Default KubeEdge version 1.6.1 is already downloaded and will checksum for it.
kubeedge-v1.6.1-linux-arm64.tar.gz checksum:
checksum_kubeedge-v1.6.1-linux-arm64.tar.gz.txt content:
kubeedge-v1.6.1-linux-arm64/
kubeedge-v1.6.1-linux-arm64/edge/
kubeedge-v1.6.1-linux-arm64/edge/edgecore
kubeedge-v1.6.1-linux-arm64/cloud/
kubeedge-v1.6.1-linux-arm64/cloud/csidriver/
kubeedge-v1.6.1-linux-arm64/cloud/csidriver/csidriver
kubeedge-v1.6.1-linux-arm64/cloud/admission/
kubeedge-v1.6.1-linux-arm64/cloud/admission/admission
kubeedge-v1.6.1-linux-arm64/cloud/cloudcore/
kubeedge-v1.6.1-linux-arm64/cloud/cloudcore/cloudcore
kubeedge-v1.6.1-linux-arm64/version
Created symlink /etc/systemd/system/multi-user.target.wants/edgecore.service → /etc/systemd/system/edgecore.service.
KubeEdge edgecore is running, For logs visit: journalctl -u edgecore.service -b
重要的提示： * –cloudcore-ipport 标志是强制性标志。 * 如果要自动为边缘节点应用证书，–token则需要。 * 云和边缘端使用的kubeEdge版本应相同。
4.3 验证
边缘端在启动edgecore后，会与云端的cloudcore进行通信，K8s进而会将边缘端作为一个node纳入K8s的管控。
$ kubectl get node
NAME STATUS ROLES AGE VERSION
chenx-latitude-3410 Ready control-plane,master 20h v1.20.5
davinci-mini Ready agent,edge 17h v1.18.6-kubeedge-v1.4.0-beta.0.242+8280082f9da41b
$ kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-7f89b7bc75-hb88t 0/1 Running 262 20h
coredns-7f89b7bc75-vh48p 0/1 Running 266 20h
etcd-chenx-latitude-3410 1/1 Running 0 20h
kube-apiserver-chenx-latitude-3410 1/1 Running 0 20h
kube-controller-manager-chenx-latitude-3410 1/1 Running 0 20h
kube-flannel-ds-6jvlr 1/1 Running 0 20h
kube-flannel-ds-r957m 0/1 Pending 0 17h
kube-proxy-bgnvp 1/1 Running 0 20h
kube-proxy-qzlxj 0/1 Pending 0 17h
kube-scheduler-chenx-latitude-3410 1/1 Running 0 20h
错误：
$ systemctl status edgecore ● edgecore.service Loaded: loaded (/etc/systemd/system/edgecore.service; enabled; vendor preset: Active: activating (auto-restart) (Result: exit-code) since Fri 2021-04-09 05 Process: 3057 ExecStart=/usr/local/bin/edgecore (code=exited, status=1/FAILURE Main PID: 3057 (code=exited, status=1/FAILURE) $ edgecore 2021-04-09 05:46:44.357072 I | INFO: Install client plugin, protocol: rest 2021-04-09 05:46:44.357414 I | INFO: Installed service discovery plugin: edge I0409 05:46:44.364335 3089 server.go:72] Version: v1.4.0-beta.0.242+8280082f9da41b I0409 05:46:44.386434 3089 sql.go:21] Begin to register twin db model I0409 05:46:44.386939 3089 module.go:34] Module twin registered successfully I0409 05:46:44.402302 3089 client.go:75] Connecting to docker on unix:///var/run/docker.sock I0409 05:46:44.402422 3089 client.go:92] Start docker client with request timeout=0s E0409 05:46:44.403689 3089 edged.go:255] init new edged error, failed to get docker version: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?原因及解决方法：
系统重启时没有docker没有自动启动，需要配置docker service 自启动或者手动启动
$ systemctl start edgecore $ systemctl status edgecore ● edgecore.service Loaded: loaded (/etc/systemd/system/edgecore.service; enabled; vendor preset: Active: active (running) since Fri 2021-04-09 05:54:57 UTC; 36s ago Main PID: 3374 (edgecore) CGroup: /system.slice/edgecore.service └─3374 /usr/local/bin/edgecore
5. Kubedge本地部署(可选)
这种方案适应于keadm工具在离线状态和部分地区由于网络原因无法使用，因此采用离线部署方式，1.3版本以后的kubeedge与之前的版本有较大改动，在密钥认证上有不兼容的情况，此处将/etc/kubeedge/ca和/etc/kubeedge/cert全部删除，在配置文件中将对应的配置注释，可成功部署。如果有公网时，建议用4. KubeEdge的安装与配置的方法。
5.1 cloud部署
5.1.1 添加device
$ cd $GOPATH/src/github.com/kubeedge/kubeedge/build/crds/devices
$ kubectl apply -f devices_v1alpha2_devicemodel.y
$ kubectl apply -f devices_v1alpha2_device.yaml
$ cd ../reliablesyncs/
$ kubectl apply -f cluster_objectsync_v1alpha1.yaml
$ kubectl apply -f objectsync_v1alpha1.yaml
5.1.2 生成并修改cloudcore.yaml配置文件
$ mkdir -p /etc/kubeedge/config && cd /etc/kubeedge/config
$ cloudcore --minconfig > /etc/kubeedge/config/cloudcore.yaml
# 清除已有的密钥
$ kubectl delete secret casecret -nkubeedge
$ kubectl delete secret cloudcoresecret -nkubeedge
# 修改生成的cloudcore.yaml文件
# 将tlsCertFile和tlsPrivateKeyFile注释，并删除/etc/kubeedge/ca 和/etc/kubeedge/cert
# 修改kubeConfig=正确的路径
# 修改advertiseAddress为对外暴露的ip
$ vim cloudcore.yaml
# With --minconfig , you can easily used this configurations as reference.
# It's useful to users who are new to KubeEdge, and you can modify/create your own configs accordingly.
# This configuration is suitable for beginners.
apiVersion: cloudcore.config.kubeedge.io/v1alpha1
kind: CloudCore
kubeAPIConfig:
kubeConfig: /root/.kube/config
master: ""
leaderelection:
LeaderElect: false
LeaseDuration: 0s
RenewDeadline: 0s
ResourceLock: ""
ResourceName: ""
ResourceNamespace: ""
RetryPeriod: 0s
modules:
cloudHub:
advertiseAddress:
- 192.168.31.5
https:
address: 0.0.0.0
enable: true
port: 10002
nodeLimit: 1000
tlsCAFile: /etc/kubeedge/ca/rootCA.crt
tlsCAKeyFile: /etc/kubeedge/ca/rootCA.key
#tlsCertFile: /etc/kubeedge/certs/stream.crt
#tlsPrivateKeyFile: /etc/kubeedge/certs/stream.key
unixsocket:
address: unix:///var/lib/kubeedge/kubeedge.sock
enable: true
websocket:
address: 0.0.0.0
enable: true
port: 10000
router:
address: 0.0.0.0
port: 9443
restTimeout: 60
5.1.3 服务自启动
# 修改 cloudcore.service
$ cd $GOPATH/src/github.com/kubeedge/kubeedge/build/tools
$ vim cloudcore.service
# 将cloudcore的路径修改为真实路径
$ cp cloudcore.service /lib/systemd/system/
$ systemctl enable cloudcore
$ systemctl start cloudcore
$ reboot
5.2 edge部署
5.2.1 生成并修改edgecore.yaml配置文件
# 不要安装kubelet或将其关闭
$ edgecore --minconfig > /etc/kubeedge/config/edgecore.yaml
# cloud端操作
$ kubectl get secret tokensecret -n kubeedge -oyaml
apiVersion: v1
data:
tokendata: YTRiNTUzYzcyNjM3MDM5NDMzMTE0MjI0OWNhNWYzNmM0N2ZhMDk3YzYwNmIwZGZlMTg2YTY3N2EzNDhkYjlmOC5leUpoYkdjaU9pSklVekkxTmlJc0luUjVjQ0k2SWtwWFZDSjkuZXlKbGVIQWlPakUyTVRnNU1ESTBOVGw5LjVNcGFRZ25ubzFnajhzZjBScjJkNGV2YzlydXpQNjd5aUtqdjAtZGhDZXc=
kind: Secret
metadata:
creationTimestamp: "2021-04-18T09:20:27Z"
managedFields:
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:data:
.: {}
f:tokendata: {}
f:type: {}
manager: cloudcore
operation: Update
time: "2021-04-18T09:20:27Z"
name: tokensecret
namespace: kubeedge
resourceVersion: "269776"
uid: 058427b5-3b9e-4839-8924-1e6edc878796
type: Opaque
# 将data.tokendata复制并进行转码
$ echo <data.tokendata> |base64 -d
# 将值填入edgecore.yaml的token处
# 修改edgecore.yaml文件
# 注释modules.edgehub.tlsCaFile,modules.edgehub.tlsCertFile,modules.edgehub.tlsPrivateKeyFile
# 修改modules.edgehub.websocket.server和modules.edgehub.quic.server
# 修改modules.edgehub.httpServer
$ vim /etc/kubeedge/config/edgecore.yaml
# With --minconfig , you can easily used this configurations as reference.
# It's useful to users who are new to KubeEdge, and you can modify/create your own configs accordingly.
# This configuration is suitable for beginners.
apiVersion: edgecore.config.kubeedge.io/v1alpha1
database:
dataSource: /var/lib/kubeedge/edgecore.db
kind: EdgeCore
modules:
edgeHub:
heartbeat: 15
httpServer: https://192.168.31.5:10002
#tlsCaFile: /etc/kubeedge/ca/rootCA.crt
#tlsCertFile: /etc/kubeedge/certs/server.crt
#tlsPrivateKeyFile: /etc/kubeedge/certs/server.key
token: "a4b553c726370394331142249ca5f36c47fa097c606b0dfe186a677a348db9f8.eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2MTg5MDE1MzN9.-fqx-7v62AO9o1iaUO0WOcgscBtyxfmTqS3oydQw4vI"
websocket:
enable: true
handshakeTimeout: 30
readDeadline: 15
server: 192.168.31.5:10000
writeDeadline: 15
edged:
cgroupDriver: cgroupfs
cgroupRoot: ""
cgroupsPerQOS: true
clusterDNS: ""
clusterDomain: ""
devicePluginEnabled: false
dockerAddress: unix:///var/run/docker.sock
gpuPluginEnabled: false
hostnameOverride: davinci-mini
nodeIP: 192.168.2.28
podSandboxImage: kubeedge/pause-arm64:3.1
remoteImageEndpoint: unix:///var/run/dockershim.sock
remoteRuntimeEndpoint: unix:///var/run/dockershim.sock
runtimeType: docker
eventBus:
mqttMode: 2
mqttQOS: 0
mqttRetain: false
mqttServerExternal: tcp://127.0.0.1:1883
mqttServerInternal: tcp://127.0.0.1:1884
# 提前导入镜像
# 如果是arm64架构，这里DLAP221是这个架构
$ docker pull kubeedge/pause-arm64:3.1
# 如果是x86_64 or amd64
$ docker pull kubeedge/pause:3.1
# 如果是arm
$ docker pull kubeedge/pause-arm:3.1
5.2.2 服务自启动
$ cd $GOPATH/src/github.com/kubeedge/kubeedge/build/tools
$ vim edgecore.service
# 将edgecore的路径修改为真实路径
$ cp edgecore.service /lib/systemd/system/
$ systemctl enable edgecore
$ reboot
5.3 部署验证
$ kubectl get node
NAME STATUS ROLES AGE VERSION
davinci-mini Ready agent,edge 46m v1.19.3-kubeedge-v1.6.1
edge-node NotReady edge 21h
hwhiaiuser-virtualbox Ready control-plane,master 2d17h v1.20.5
$ kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-7f89b7bc75-jr5cm 1/1 Running 2 2d17h
coredns-7f89b7bc75-pv978 1/1 Running 2 2d17h
etcd-hwhiaiuser-virtualbox 1/1 Running 2 2d17h
kube-apiserver-hwhiaiuser-virtualbox 1/1 Running 2 2d17h
kube-controller-manager-hwhiaiuser-virtualbox 1/1 Running 2 2d17h
kube-flannel-ds-bhcsq 1/1 Running 2 2d17h
kube-flannel-ds-cchzh 0/1 Pending 0 47m
kube-proxy-l64tq 1/1 Running 2 2d17h
kube-proxy-p222c 0/1 Pending 0 47m
kube-scheduler-hwhiaiuser-virtualbox 1/1 Running 2 2d17h
$ kubectl get no -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
davinci-mini Ready agent,edge 49m v1.19.3-kubeedge-v1.6.1 192.168.2.28 <none> Ubuntu 18.04.4 LTS 4.19.95+ docker://20.10.2
edge-node NotReady edge 21h <none> <none> <unknown> <unknown> <unknown>
hwhiaiuser-virtualbox Ready control-plane,master 2d17h v1.20.5 192.168.31.5 <none> Ubuntu 18.04.5 LTS 4.15.0-142-generic docker://20.10.2
6 Kubedge二进制部署(可选)
6.1 Cloud部署
6.1.1 准备文件
#从官网下载keadm-v1.6.1-linux-amd64.tar.gz， 并解压缩，
$ cd ~/keadm-v1.6.1-linux-amd64/keadm
如果以前部署过，则reset前期部署
$ ./keadm reset
#分别从官网下载以下文件，并放在/etc/kubeedge目录，没有此目录则创建该目录，建议下载对应版本的源代码，从源代码中提取crds和cloudcore.service,如何获取源代码，可以参见4. KubeEdge的安装与配置
root@chenx-Latitude-3410:/etc/kubeedge# tree
.
├── checksum_kubeedge-v1.6.1-linux-amd64.tar.gz.txt
├── cloudcore.service
├── crds
│ ├── devices
│ │ ├── devices_v1alpha1_devicemodel.yaml
│ │ ├── devices_v1alpha1_device.yaml
│ │ ├── devices_v1alpha2_devicemodel.yaml
│ │ └── devices_v1alpha2_device.yaml
│ ├── reliablesyncs
│ │ ├── cluster_objectsync_v1alpha1.yaml
│ │ └── objectsync_v1alpha1.yaml
│ └── router
│ ├── router_v1_ruleEndpoint.yaml
│ └── router_v1_rule.yaml
└── kubeedge-v1.6.1-linux-amd64.tar.gz
4 directories, 11 files
6.1.2 部署操作
$ cd ~/keadm-v1.6.1-linux-amd64/keadm
$ ./keadm init --advertise-address="192.168.31.4" --kubeedge-version="1.6.1"
Kubernetes version verification passed, KubeEdge installation will start...
W0421 17:11:57.247298 4449 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
W0421 17:11:57.358400 4449 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
W0421 17:11:57.394461 4449 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
W0421 17:11:57.457284 4449 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
Expected or Default KubeEdge version 1.6.1 is already downloaded and will checksum for it.
kubeedge-v1.6.1-linux-amd64.tar.gz checksum:
checksum_kubeedge-v1.6.1-linux-amd64.tar.gz.txt content:
kubeedge-v1.6.1-linux-amd64.tar.gz in your path checksum failed and do you want to delete this file and try to download again?
[y/N]: #如果提示checksum不对，可以选择N，忽略它
n
W0421 17:12:15.561640 4449 common.go:276] failed to checksum and will continue to install.
[Run as service] service file already exisits in /etc/kubeedge//cloudcore.service, skip download
kubeedge-v1.6.1-linux-amd64/
kubeedge-v1.6.1-linux-amd64/edge/
kubeedge-v1.6.1-linux-amd64/edge/edgecore
kubeedge-v1.6.1-linux-amd64/cloud/
kubeedge-v1.6.1-linux-amd64/cloud/csidriver/
kubeedge-v1.6.1-linux-amd64/cloud/csidriver/csidriver
kubeedge-v1.6.1-linux-amd64/cloud/admission/
kubeedge-v1.6.1-linux-amd64/cloud/admission/admission
kubeedge-v1.6.1-linux-amd64/cloud/cloudcore/
kubeedge-v1.6.1-linux-amd64/cloud/cloudcore/cloudcore
kubeedge-v1.6.1-linux-amd64/version
KubeEdge cloudcore is running, For logs visit: /var/log/kubeedge/cloudcore.log
CloudCore started
root@chenx-Latitude-3410:/home/HwHiAiUser/kubeedge/keadm-v1.6.1-linux-amd64/keadm#
6.1.3 验证
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
chenx-latitude-3410 Ready control-plane,master 9m50s v1.20.5
$ systemctl status cloudcore
Unit cloudcore.service could not be found. #如果没有发现cloudcore服务，则自行手动创建
$ cp /etc/kubeedge/cloudcore.service /lib/systemd/system/
$ systemctl enable cloudcore
#查看cloudcore是否被启动了
$ ps aux | grep cloudcore
root 4611 0.2 0.7 1265340 46792 pts/0 Sl 17:12 0:00 /usr/local/bin/cloudcore
root 6529 0.0 0.0 16184 1020 pts/0 S+ 17:16 0:00 grep --color=auto cloudcore
$ kill -9 4611
$ ps aux | grep cloudcore
root 8178 0.0 0.0 16184 1044 pts/0 S+ 17:20 0:00 grep --color=auto cloudcore
$ systemctl start cloudcore
$ systemctl status cloudcore
● cloudcore.service
Loaded: loaded (/lib/systemd/system/cloudcore.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2021-04-21 17:20:13 CST; 24s ago #看到active(running),表示启动正常
Main PID: 8267 (cloudcore)
Tasks: 9 (limit: 4915)
CGroup: /system.slice/cloudcore.service
└─8267 /usr/local/bin/cloudcore
4月 21 17:20:14 chenx-Latitude-3410 cloudcore[8267]: I0421 17:20:14.058041 8267 upstream.go:110] start upstream controller
4月 21 17:20:14 chenx-Latitude-3410 cloudcore[8267]: I0421 17:20:14.070571 8267 downstream.go:870] Start downstream devicecontroller
4月 21 17:20:14 chenx-Latitude-3410 cloudcore[8267]: I0421 17:20:14.071107 8267 downstream.go:550] start downstream controller
4月 21 17:20:14 chenx-Latitude-3410 cloudcore[8267]: I0421 17:20:14.158101 8267 server.go:269] Ca and CaKey don't exist in local directory, and will read from the secret
4月 21 17:20:14 chenx-Latitude-3410 cloudcore[8267]: I0421 17:20:14.168090 8267 server.go:317] CloudCoreCert and key don't exist in local directory, and will read from the secret
4月 21 17:20:14 chenx-Latitude-3410 cloudcore[8267]: I0421 17:20:14.198254 8267 signcerts.go:98] Succeed to creating token
4月 21 17:20:14 chenx-Latitude-3410 cloudcore[8267]: I0421 17:20:14.199135 8267 server.go:44] start unix domain socket server
4月 21 17:20:14 chenx-Latitude-3410 cloudcore[8267]: I0421 17:20:14.199641 8267 uds.go:72] listening on: //var/lib/kubeedge/kubeedge.sock
4月 21 17:20:14 chenx-Latitude-3410 cloudcore[8267]: I0421 17:20:14.221045 8267 server.go:63] Starting cloudhub websocket server
4月 21 17:20:16 chenx-Latitude-3410 cloudcore[8267]: I0421 17:20:16.076248 8267 upstream.go:63] Start upstream devicecontroller
$ kubectl get secret -A |grep secret
kubeedge casecret Opaque 2 5d22h
kubeedge cloudcoresecret Opaque 2 5d22h
kubeedge tokensecret Opaque 1 5d22h
6.2 Edge部署
6.2.1 文件准备
#从官网下载keadm-v1.6.1-linux-arm64.tar.gz
$ tar xvf keadm-v1.6.1-linux-arm64.tar.gz
$ tree keadm-v1.6.1-linux-arm64
keadm-v1.6.1-linux-arm64
├── keadm
│ └── keadm
└── version
1 directory, 2 files
如果以前部署过Edge,运行以下命令，reset以前的部署
$ cd keadm-v1.6.1-linux-arm64/keadm
$ ./keadm reset
#从官网下载以下文件，放在/etc/kubeedge, 如果没有此目录，则创建
#checksum_kubeedge-v1.6.1-linux-arm64.tar.gz.txt
#edgecore.service, 此文件建议从对应版本的源代码中提取
#kubeedge-v1.6.1-linux-arm64.tar.gz
$ tree /etc/kubeedge
/etc/kubeedge
├── checksum_kubeedge-v1.6.1-linux-arm64.tar.gz.txt
├── edgecore.service
└── kubeedge-v1.6.1-linux-arm64.tar.gz
0 directories, 3 files
6.2.2 部署操作
#在云端运行以下命令获取token
$ cd <keadm所在的目录>
$ keadm gettoken
5b16a62bf36e4523a89186db65a8c6f569fe5f1403cd1a6e040cedb2ea7e250d.eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2MTkwODMyMTR9.D086j9BIGVNdFzvCmeetsBVSRd6R-fo7jbkf70R6nHU
$ cd ~/keadm-v1.6.1-linux-arm64/keadm
$ ./keadm join --cloudcore-ipport=192.168.31.4:10000 --token=5b16a62bf36e4523a89186db65a8c6f569fe5f1403cd1a6e040cedb2ea7e250d.eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2MTkwODMyMTR9.D086j9BIGVNdFzvCmeetsBVSRd6R-fo7jbkf70R6nHU --kubeedge-version="1.6.1"
Host has mosquit+ already installed and running. Hence skipping the installation steps !!!
Expected or Default KubeEdge version 1.6.1 is already downloaded and will checksum for it.
kubeedge-v1.6.1-linux-arm64.tar.gz checksum:
checksum_kubeedge-v1.6.1-linux-arm64.tar.gz.txt content:
Expected or Default KubeEdge version 1.6.1 is already downloaded
[Run as service] service file already exisits in /etc/kubeedge//edgecore.service, skip download
kubeedge-v1.6.1-linux-arm64/
kubeedge-v1.6.1-linux-arm64/edge/
kubeedge-v1.6.1-linux-arm64/edge/edgecore
kubeedge-v1.6.1-linux-arm64/cloud/
kubeedge-v1.6.1-linux-arm64/cloud/csidriver/
kubeedge-v1.6.1-linux-arm64/cloud/csidriver/csidriver
kubeedge-v1.6.1-linux-arm64/cloud/admission/
kubeedge-v1.6.1-linux-arm64/cloud/admission/admission
kubeedge-v1.6.1-linux-arm64/cloud/cloudcore/
kubeedge-v1.6.1-linux-arm64/cloud/cloudcore/cloudcore
kubeedge-v1.6.1-linux-arm64/version
KubeEdge edgecore is running, For logs visit: journalctl -u edgecore.service -b
root@davinci-mini:~/keadm-v1.6.1-linux-arm64/keadm#
6.2.3 验证
Edge端
$ systemctl status edgecore
● edgecore.service
Loaded: loaded (/etc/systemd/system/edgecore.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2021-04-21 09:57:29 UTC; 14min ago #active(running)表示运行正常
Main PID: 6544 (edgecore)
CGroup: /system.slice/edgecore.service
└─6544 /usr/local/bin/edgecore
root@davinci-mini:~/keadm-v1.6.1-linux-arm64/keadm#
Cloud端
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
chenx-latitude-3410 Ready control-plane,master 67m v1.20.5
davinci-mini Ready agent,edge 15m v1.19.3-kubeedge-v1.6.1
7. 容器应用部署
装好之后，尝试做一个应用部署，来做三件事情：
- 提交一个 nginx deployment；
- 升级 nginx deployment；
- 扩容 nginx deployment
我们提交一个 nginx 的 Deployment，然后对这个 Deployment 进行一次版本升级，也就是改变它中间 Pod 的版本。最后我们也会尝试对 nginx 进行一次扩容，进行一次水平的伸缩，下面就让大家一起跟我来尝试这三个操作吧。
7.1 提交部署
我们利用 kubectl 来看一下这个集群中节选的状态，可以看到这个 master 的节点已经是 Ready 状态：
$ kubectl get node
NAME STATUS ROLES AGE VERSION
chenx-latitude-3410 Ready control-plane,master 20h v1.20.5
davinci-mini Ready agent,edge 17h v1.18.6-kubeedge-v1.4.0-beta.0.242+8280082f9da41b
我们就以这个为节点，下面我们尝试去看一下现在集群中 Deployment 这个资源：
$ kubectl get deployments
No resources found in default namespace.
可以看到集群中没有任何的 Deployment.
现在做的第一个操作是去创建一个 Deployment。可以看到下面的deployment.yaml文件，这是一个 API 的 content，它的 kind 是 Deployment，name 是 nginx-deployment, 它的 replicas 数目是 2，它的镜像版本是 1.14.2。
#https://k8s.io/examples/application/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
selector:
matchLabels:
app: nginx
replicas: 2 # tells deployment to run 2 pods matching the template
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
我们下面还是回到 kubectl 这个 commnd 来执行这次 Deployment 的真正的操作。我们可以看到一个简单的操作，就会去让 Deployment 生成2个副本。
$ kubectl apply -f https://k8s.io/examples/application/deployment.yaml
deployment.apps/nginx-deployment created
下面也可以 describe 一下现在的 Deployment 的状态。我们知道之前是没有这个 Deployment 的，现在我们去 describe 这个 nginx-deployment。
$ kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deployment 0/2 2 0 57s
$ kubectl describe deployment nginx-deployment
Name: nginx-deployment
Namespace: default
CreationTimestamp: Fri, 09 Apr 2021 14:32:55 +0800
Labels: <none>
Annotations: deployment.kubernetes.io/revision: 1
Selector: app=nginx
Replicas: 2 desired | 2 updated | 2 total | 2 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: app=nginx
Containers:
nginx:
Image: nginx:1.14.2
Port: 80/TCP
Host Port: 0/TCP
Environment: <none>
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
Available True MinimumReplicasAvailable
Progressing True NewReplicaSetAvailable
OldReplicaSets: <none>
NewReplicaSet: nginx-deployment-66b6c48dd5 (2/2 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
我们可以看到：有一个 nginx-deployment 已经被生成了，它的 replicas 数目也是我们想要的、selector 也是我们想要的、它的 image 的版本也是1.14.2。
我们在端侧看看，应用是否已被部署
root@davinci-mini:/lib/systemd/system# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
08ac609cefca 7bbc8783b8ec "nginx -g 'daemon of…" 3 minutes ago Up 3 minutes k8s_nginx_nginx-deployment-66b6c48dd5-swwvf_default_b64cecaa-2057-48ad-aacb-235832e8f769_0
4ef355b3c43d kubeedge/pause-arm64:3.1 "/pause" 3 minutes ago Up 3 minutes k8s_POD_nginx-deployment-66b6c48dd5-swwvf_default_b64cecaa-2057-48ad-aacb-235832e8f769_0
63f73a26ef69 nginx "nginx -g 'daemon of…" 3 minutes ago Up 3 minutes k8s_nginx_nginx-deployment-66b6c48dd5-xp45h_default_f61ce465-d4e1-4c38-b500-860faad453a4_0
b88a26c9e7eb kubeedge/pause-arm64:3.1 "/pause" 4 minutes ago Up 4 minutes k8s_POD_nginx-deployment-66b6c48dd5-xp45h_default_f61ce465-d4e1-4c38-b500-860faad453a4_0
我们可以看到容器应用已经成功部署到了端侧，而且是我们设置的两个副本。
7.2 升级部署
下面我们去升级这个 Deployment 版本，首先下载另外一个 yaml 文件 deployment-update.yaml，可以看到这里面的 image 本身的版本号从1.14.2升级到 1.16.1。
#https://k8s.io/examples/application/deployment-update.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
selector:
matchLabels:
app: nginx
replicas: 2
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.16.1 # Update the version of nginx from 1.14.2 to 1.16.1
ports:
- containerPort: 80
接下来我们重新 apply 新的 deployment-update 这个 yaml 文件。
$ kubectl apply -f https://k8s.io/examples/application/deployment-update.yaml
我们可以 discribe 具体去看一下是不是所有 Pod 的版本都被更新了，可以看到这个 image 的版本由 1.14.2 真正更新到了 1.16.1。
$ kubectl describe deployment nginx-deployment
Name: nginx-deployment
Namespace: default
CreationTimestamp: Fri, 09 Apr 2021 14:32:55 +0800
Labels: <none>
Annotations: deployment.kubernetes.io/revision: 2
Selector: app=nginx
Replicas: 2 desired | 2 updated | 2 total | 2 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: app=nginx
Containers:
nginx:
Image: nginx:1.16.1
Port: 80/TCP
Host Port: 0/TCP
Environment: <none>
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
Available True MinimumReplicasAvailable
Progressing True NewReplicaSetAvailable
OldReplicaSets: <none>
NewReplicaSet: nginx-deployment-559d658b74 (2/2 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 17m deployment-controller Scaled up replica set nginx-deployment-66b6c48dd5 to 2
Normal ScalingReplicaSet 11m deployment-controller Scaled up replica set nginx-deployment-559d658b74 to 1
Normal ScalingReplicaSet 10m deployment-controller Scaled down replica set nginx-deployment-66b6c48dd5 to 1
Normal ScalingReplicaSet 10m deployment-controller Scaled up replica set nginx-deployment-559d658b74 to 2
Normal ScalingReplicaSet 9m53s deployment-controller Scaled down replica set nginx-deployment-66b6c48dd5 to 0
最后，我们也可以看到 controller 又执行了几次新的操作，这个控制器维护了整个 Deployment 和 Pod 状态。
7.3 应用扩容
最后我们演示一下给 Deployment 做水平扩张，下载另一个 yaml 文件 deployment-scale.yaml，这里面的 replicas 数目已经从 2 改成了 4, 而且将版本号也变成了1.14.2.
#https://kubernetes.io/examples/application/deployment-scale.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
selector:
matchLabels:
app: nginx
replicas: 4 # Update the replicas from 2 to 4
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
利用 kubectl 去 apply 这个新的 deployment-scale.yaml 文件，
$ kubectl apply -f https://kubernetes.io/examples/application/deployment-scale.yaml
当我们执行了 deployment-scale 操作之后，它的容器 Pod 数目从 2 变成了 4。我们可以再一次 describ 一下当前集群中的 deployment 的情况，可以看到它的 replicas 的数目从 2 变到了 4，同时也可以看到 controller 又做了几次新的操作，这个 scale up 成功了
$ kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deployment 4/4 4 4 22m
root@chenx-Latitude-3410:~# kubectl describe deployment nginx-deployment
Name: nginx-deployment
Namespace: default
CreationTimestamp: Fri, 09 Apr 2021 14:32:55 +0800
Labels: <none>
Annotations: deployment.kubernetes.io/revision: 3
Selector: app=nginx
Replicas: 4 desired | 4 updated | 4 total | 4 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: app=nginx
Containers:
nginx:
Image: nginx:1.14.2
Port: 80/TCP
Host Port: 0/TCP
Environment: <none>
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
Available True MinimumReplicasAvailable
Progressing True NewReplicaSetAvailable
OldReplicaSets: <none>
NewReplicaSet: nginx-deployment-66b6c48dd5 (4/4 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 16m deployment-controller Scaled up replica set nginx-deployment-559d658b74 to 1
Normal ScalingReplicaSet 14m deployment-controller Scaled down replica set nginx-deployment-66b6c48dd5 to 1
Normal ScalingReplicaSet 14m deployment-controller Scaled up replica set nginx-deployment-559d658b74 to 2
Normal ScalingReplicaSet 14m deployment-controller Scaled down replica set nginx-deployment-66b6c48dd5 to 0
Normal ScalingReplicaSet 30s deployment-controller Scaled down replica set nginx-deployment-559d658b74 to 3
Normal ScalingReplicaSet 30s deployment-controller Scaled up replica set nginx-deployment-559d658b74 to 4
Normal ScalingReplicaSet 30s deployment-controller Scaled up replica set nginx-deployment-66b6c48dd5 to 1
Normal ScalingReplicaSet 30s (x2 over 22m) deployment-controller Scaled up replica set nginx-deployment-66b6c48dd5 to 2
Normal ScalingReplicaSet 24s deployment-controller Scaled down replica set nginx-deployment-559d658b74 to 2
Normal ScalingReplicaSet 24s deployment-controller Scaled up replica set nginx-deployment-66b6c48dd5 to 3
Normal ScalingReplicaSet 24s deployment-controller Scaled down replica set nginx-deployment-559d658b74 to 1
Normal ScalingReplicaSet 24s deployment-controller Scaled up replica set nginx-deployment-66b6c48dd5 to 4
Normal ScalingReplicaSet 14s deployment-controller Scaled down replica set nginx-deployment-559d658b74 to 0
在端侧，我们也看到4个副本在运行
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
038ddfa21a07 7bbc8783b8ec "nginx -g 'daemon of…" 54 minutes ago Up 54 minutes k8s_nginx_nginx-deployment-66b6c48dd5-dr9sb_default_b3c93ef6-a799-427f-abb5-e1261d07ac4f_0
e563079ee081 7bbc8783b8ec "nginx -g 'daemon of…" 54 minutes ago Up 54 minutes k8s_nginx_nginx-deployment-66b6c48dd5-79j9n_default_25c4860b-3156-4b1e-9bb1-f1114207e7ad_0
459406dd377c kubeedge/pause-arm64:3.1 "/pause" 54 minutes ago Up 54 minutes k8s_POD_nginx-deployment-66b6c48dd5-dr9sb_default_b3c93ef6-a799-427f-abb5-e1261d07ac4f_0
e023187a1ea3 kubeedge/pause-arm64:3.1 "/pause" 54 minutes ago Up 54 minutes k8s_POD_nginx-deployment-66b6c48dd5-79j9n_default_25c4860b-3156-4b1e-9bb1-f1114207e7ad_0
20e7bd1cbcd1 7bbc8783b8ec "nginx -g 'daemon of…" 54 minutes ago Up 54 minutes k8s_nginx_nginx-deployment-66b6c48dd5-thmz2_default_1f6d3bde-4b08-47a9-9f1a-1db97d6cfdb4_0
bd1de8065887 7bbc8783b8ec "nginx -g 'daemon of…" 54 minutes ago Up 54 minutes k8s_nginx_nginx-deployment-66b6c48dd5-jfz4v_default_be61f03c-1ca4-4041-b795-a783578f9d6d_0
813ce4c20f67 kubeedge/pause-arm64:3.1 "/pause" 54 minutes ago Up 54 minutes k8s_POD_nginx-deployment-66b6c48dd5-thmz2_default_1f6d3bde-4b08-47a9-9f1a-1db97d6cfdb4_0
11b73387bf0a kubeedge/pause-arm64:3.1 "/pause" 54 minutes ago Up 54 minutes k8s_POD_nginx-deployment-66b6c48dd5-jfz4v_default_be61f03c-1ca4-4041-b795-a783578f9d6d_0
1382604b7b8a kubeedge/pause-arm64:3.1 "/pause" 54 minutes ago Up 54 minutes k8s_POD_nginx-deployment-559d658b74-z6tkh_default_830c8ff8-9e32-4bc9-8eb4-c1f5e7439835_0
7.4 删除应用部署
最后，让我们利用 delete 操作把我们刚才生成的 Deployment 给删除掉。kubectl delete deployment，也是刚才我们本身的 deployment name，当我们把它删除掉之后，我们所有的操作就完成了。
$ kubectl delete deployment nginx-deployment
deployment.apps "nginx-deployment" deleted
$ kubectl get deployments
No resources found in default namespace.
我们再去重新 get 这个 Deployment，也会显示这个资源不再存在，这个集群又回到了最开始干净的状态。
在端侧，我们也发现应用被删除了
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1382604b7b8a kubeedge/pause-arm64:3.1 "/pause" About an hour ago Up About an hour k8s_POD_nginx-deployment-559d658b74-z6tkh_default_830c8ff8-9e32-4bc9-8eb4-c1f5e7439835_0