基础配置
这里提供一种基于Kubeadm快速部署K8s集群的操作方式，基础环境如下：
| ip | os | component | hostname | Cpu | Memory | Storage |
|---|---|---|---|---|---|---|
| 172.30.3.220 | Ubuntu16.04 server | master | k8s-n1 | 4 Core | 8G | 200G |
| 172.30.3.221 | Ubuntu16.04 server | node | k8s-n2 | 4 Core | 8G | 200G |
| 172.30.3.222 | Ubuntu16.04 server | node | k8s-n3 | 4 Core | 8G | 200G |
kubelet版本：V1.14
Docker版本：V18.09.2
没有特别说明，用户使用root
首先，三台机器配置好hosts并关闭sawp：
# 三台机器的hosts文件中添加：
172.30.3.220 k8s-n1
172.30.3.221 k8s-n2
172.30.3.222 k8s-n3
# 关闭swap
$ swapoff -a
其次，给三台机器配置ssh免密码登录，非必要步骤，主要是方便操作。
# k8s-n1、k8s-n2、k8s-n3生成ssh密钥
$ ssh-keygen -t rsa
# 创建~/.ssh/authorized_keys
$ touch ~/.ssh/authorized_keys
# 将公钥导入到授权keys
$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
# 将k8s-n1的授权keys发送给k8s-n2
$ scp -i ~/.ssh/authorized_keys root@k8s-n2:~/.ssh/
# 登录到k8s-n2并将公钥追加到授权keys
$ ssh root@k8s-n2
$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
# 将k8s-n2的授权keys发送给k8s-n3
$ scp -i ~/.ssh/authorized_keys root@k8s-n3:~/.ssh/
# 登录到k8s-n3并将公钥追加到授权keys
$ ssh root@k8s-n3
$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
# 将k8s-n3上的授权keys文件发送到k8s-n1、k8s-n2
$ scp -i ~/.ssh/authorized_keys root@k8s-n1:~/.ssh/
$ scp -i ~/.ssh/authorized_keys root@k8s-n2:~/.ssh/
给三台机器安装Docker：
# 更新apt源（可以翻墙的情况下，直接使用ubuntu的原声配置就阔以）
$ apt update
# 安装最新的docker
$ apt install docker.io -y
# 设置开机启动，并启动服务
$ systemctl enable docker && systemctl start docker
# 查看docker是否启动成功
$ systemctl status docker
# 如果启动失败，可以通过该命令查看启动过程
$ journalctl -xefu docker
给三台机器配置kubelet、kubeadm、kubectl等组件：
# 安装必要组建
$ apt-get update && apt-get install -y apt-transport-https
# 配置证书和deb源
$ curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
$ cat </etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF
# 更新源并安装
$ apt-get update
$ apt-get install -y kubelet kubeadm kubectl
$ systemctl enable kubelet
$ systemctl start kubelet
# 注意，这个时候kubelet还不能正常启动，因为缺少基础配置文件
# 通过如下指令查看不能启动的原因
$ journalctl -xefu kubelet
Maser节点K8s-n1的配置
开始配置Master节点K8s-n1：
# 配置cgroup driver，先看下默认配置，一般是cgroupfs
$ docker info | grep cgroup
# 再看下kube10-kubeadm.conf中的--cgroup-driver 默认是 systemd
# 在这一行的后面添加"--cgroup-driver=cgroupfs"
$ vi /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
# 变成
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --cgroup-driver=cgroupfs"
# 重启kubelet
$ systemctl daemon-reload
$ systemctl restart kubelet
初始化master节点：
# --pod-network-cidr是网络插件flannel会用到的参数，设定pod会使用的IP range
$ kubeadm init --apiserver-advertise-address=172.30.3.220 --pod-network-cidr=10.244.0.0/16
# 命令执行成功后会输出join node的指令：
$ kubeadm join 172.30.3.220:6443 --token jkuc3w.5sq85b4dh5f2deet \
--discovery-token-ca-cert-hash sha256:ad953ebdc367105595ec70b9d4d9d2a17cc6c98e68bd0b8857bce34745c1a9d5
# 当需要添加node节点时，可以在node机器上运行该指令
# 如果失败，可以运行以下指令重来：
$ kubeadm reset
此时，k8s-n1上的kubelet服务应该是正常的状态，但是提示缺少网络插件：
$ systemctl status kubelet
kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: active (running) since Wed 2019-04-10 21:59:46 EDT; 3h 2min ago
Docs: https://kubernetes.io/docs/home/
Main PID: 31544 (kubelet)
Tasks: 18
Memory: 47.7M
CPU: 32min 52.401s
CGroup: /system.slice/kubelet.service
└─31544 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml
Apr 11 01:02:40 k8s-n1 kubelet[31544]: E0411 01:02:40.342504 31544 remote_runtime.go:109] RunPodSandbox from runtime service failed: rpc error: code = Unknown desc = failed to
Apr 11 01:02:40 k8s-n1 kubelet[31544]: E0411 01:02:40.342633 31544 kuberuntime_sandbox.go:68] CreatePodSandbox for pod "coredns-fb8b8dccf-tx5vf_kube-system(b9daf3d4-5c04-11e9-9
Apr 11 01:02:40 k8s-n1 kubelet[31544]: E0411 01:02:40.342724 31544 kuberuntime_manager.go:693] createPodSandbox for pod "coredns-fb8b8dccf-tx5vf_kube-system(b9daf3d4-5c04-11e9-
Apr 11 01:02:40 k8s-n1 kubelet[31544]: E0411 01:02:40.342941 31544 pod_workers.go:190] Error syncing pod b9daf3d4-5c04-11e9-99bc-000c2968fc47 ("coredns-fb8b8dccf-tx5vf_kube-sys
Apr 11 01:02:40 k8s-n1 kubelet[31544]: W0411 01:02:40.437223 31544 docker_sandbox.go:384] failed to read pod IP from plugin/docker: NetworkPlugin cni failed on the status hook
Apr 11 01:02:40 k8s-n1 kubelet[31544]: W0411 01:02:40.494044 31544 pod_container_deletor.go:75] Container "7ab3628dd31222b83406e4366847eaed26d8b5def60bd5d4a87700b215b25e0a" not
Apr 11 01:02:40 k8s-n1 kubelet[31544]: W0411 01:02:40.498937 31544 cni.go:309] CNI failed to retrieve network namespace path: cannot find network namespace for the terminated c
Apr 11 01:02:40 k8s-n1 kubelet[31544]: W0411 01:02:40.516730 31544 docker_sandbox.go:384] failed to read pod IP from plugin/docker: NetworkPlugin cni failed on the status hook
Apr 11 01:02:40 k8s-n1 kubelet[31544]: W0411 01:02:40.566554 31544 pod_container_deletor.go:75] Container "64449655fad7b8397bc62582bccb81cd77998b173a879b354ca29b882f4a30de" not
Apr 11 01:02:40 k8s-n1 kubelet[31544]: W0411 01:02:40.571572 31544 cni.go:309] CNI failed to retrieve network namespace path: cannot find network namespace for the terminated c
配置网络flannel
这是正常的，因为k8s集群网络必须依赖外部的网络插件，接下来安装网络插件flannel ：
$ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
podsecuritypolicy.extensions/psp.flannel.unprivileged created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.extensions/kube-flannel-ds-amd64 created
daemonset.extensions/kube-flannel-ds-arm64 created
daemonset.extensions/kube-flannel-ds-arm created
daemonset.extensions/kube-flannel-ds-ppc64le created
daemonset.extensions/kube-flannel-ds-s390x created
# 查看安装情况
$ kubeclt -n kube-system get pod
NAME READY STATUS RESTARTS AGE
coredns-fb8b8dccf-dthq9 1/1 Running 0 158m
coredns-fb8b8dccf-tx5vf 1/1 Running 0 158m
etcd-k8s-n1 1/1 Running 0 3h28m
kube-apiserver-k8s-n1 1/1 Running 0 3h28m
kube-controller-manager-k8s-n1 1/1 Running 0 3h28m
kube-flannel-ds-amd64-zrn9h 1/1 Running 0 32s
kube-proxy-6vswp 1/1 Running 0 3h29m
kube-scheduler-k8s-n1 1/1 Running 0 3h28m
# 也可以看到flannel的镜像
$ docker image ls |grep flannel
quay.io/coreos/flannel v0.11.0-amd64 ff281650a721 2 months ago 52.6MB
配置node节点(添加/删除Node节点)
分别在k8s-n2、k8s-n3上执行：
# 添加node节点
$ kubeadm join 172.30.3.220:6443 --token cvenka.7z3kqwjo9ca6k4js \
--discovery-token-ca-cert-hash sha256:55e84a80cd67fb070d1484a5d28e1457cdcbe5f4d781fc8ec78de5843fce6cc8
# 删除node k8s-n2节点
$ kubectl drain k8s-n2 --delete-local-data --force --ignore-daemonsets
$ kubectl delete node k8s-n2
# 在k8s-n2节点重置
$ kubeadm reset
配置kubectl访问k8s集群
# 配置环境变量
vi /etc/profile
# 添加
export KUBECONFIG=/etc/kubernetes/admin.conf
# 配置非root用户访问，非必要过程
$ su yzhou
$ mkdir -p ～/.kube
$ sudo cp -i /etc/kubernetes/admin.conf ～/.kube/config
$ sudo chown $(id -u):$(id -g) ～/.kube/config
# 配置本地用户访问(macOS 安装kubectl)
$ brew install kubectl
$ mkdir ~/.kube
$ vim ~/.kube/config
# 把k8s-n1机器上的/etc/kubernetes/admin.conf文件的内容copy出来复制到本地的～/.kube/config
$ sudo chown $(id -u):$(id -g) ～/.kube/config
# 测试kubectl
$ kubectl cluster-info
# 有结果输出，无错误则配置正确，如果失败，使用以下指令查看日志
$ kubectl cluster-info dump
查看集群状态
# 查看node情况
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-n1 Ready master 3h v1.14.1
k8s-n2 Ready 3m v1.14.1
k8s-n3 Ready 1m v1.14.1
# 查看kube的系统pod情况
$ kubectl get pods -o wide --all-namespaces=true
NAME READY STATUS RESTARTS AGE
coredns-fb8b8dccf-dthq9 1/1 Running 0 2h
coredns-fb8b8dccf-tx5vf 1/1 Running 0 2h
etcd-k8s-n1 1/1 Running 0 3h
kube-apiserver-k8s-n1 1/1 Running 0 3h
kube-controller-manager-k8s-n1 1/1 Running 0 3h
kube-flannel-ds-amd64-d5rqq 1/1 Running 0 4m
kube-flannel-ds-amd64-sqfqt 1/1 Running 0 2m
kube-flannel-ds-amd64-zrn9h 1/1 Running 0 10m
kube-proxy-2bhnb 1/1 Running 0 4m
kube-proxy-6vswp 1/1 Running 0 3h
kube-proxy-bcnrb 1/1 Running 0 2m
kube-scheduler-k8s-n1 1/1 Running 0 3h
到此为止，基于kubeadm搭建的k8s集群就全部完毕了。