安装kubernetes1.14.0
此手册适用于单master，以及etcd部署在master上的环境。
部署要求：
| 操作系统 | docker要求操作系统内核在3.10以上，uname -a | 3.10.0-957.21.3.el7.x86_64 |
|---|---|---|
| cpu和内存 | master：至少2core，4GB内存；node至少4core，16GB内存 | |
| etcd | 3.0版本及以上 | |
| docker | 18.03版本及以上 | Version: 18.09.7 |
环境上安装的组件：
| master | kubelet-1.14.0-0.x86_64， kubeadm-1.14.0-0.x86_64， kubectl-1.14.0-0.x86_64，docker 18.09.7 |
|---|---|
| node | kubelet-1.14.0-0.x86_64， kubeadm-1.14.0-0.x86_64， kubectl-1.14.0-0.x86_64，docker 18.09.7 |
--------------------------------以下操作在所有节点执行----------------------------------
1、关闭防火墙
systemctl status firewalld
systemctl disable firewalld
systemctl stop firewalld
2、关闭selinux
sed -i 's/^SELINUX=enforcing$/SELINUX=disabled/' /etc/selinux/config && setenforce 0
3、永久关闭swap
swapoff -a
cp /etc/fstab /etc/fstab_bak
cat /etc/fstab_bak | grep -v swap > /etc/fstab
4、配置时间同步
使用chrony同步时间，centos7默认已安装，这里修改时钟源，所有节点与网络时钟源同步
1） 安装chrony：
yum install -y chrony
cp /etc/chrony.conf{,.bak}
2）注释默认ntp服务器
sed -i 's/^server/#&/' /etc/chrony.conf
3）指定上游公共 ntp 服务器
cat >> /etc/chrony.conf << EOF
server 0.asia.pool.ntp.org iburst
server 1.asia.pool.ntp.org iburst
server 2.asia.pool.ntp.org iburst
server 3.asia.pool.ntp.org iburst
EOF
4） 设置时区
timedatectl set-timezone Asia/Shanghai
5）重启chronyd服务并设为开机启动：
systemctl enable chronyd && systemctl restart chronyd
6）验证,查看当前时间以及存在带*的行
timedatectl && chronyc sources
5、配置内核参数
cat > /etc/sysctl.d/k8s.conf <6、安装docker
1）下载docker-ce官方的yum源配置文件
[root@localhost ~]# yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
2）禁用docker-c-edge源配edge是不开发版，不稳定，下载stable版
yum-config-manager --disable docker-ce-edge
3）更新本地YUM源缓存
yum makecache fast
4）安装Docker-ce相应版本的
yum -y install docker-ce
7、在所有节点上安装kubeadm
1）配置阿里源
cat < /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes Repository
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
EOF
2）安装kubeadm、kubelet、kubectl，一定要标注版本，因为后面下载kubernetes相关镜像的时候指定了1.14版本，如果不指定版本，kubelet会默认下载最新的，当kubelet版本高于kubernetes时，会报错。
yum install -y kubelet-1.14.0-0.x86_64 kubeadm-1.14.0-0.x86_64 kubectl-1.14.0-0.x86_64 --disableexcludes=kubernetes
–disableexcludes=kubernetes代表使用kubernetes这个repo
3）初始化kubelet，并加入开机自动启动
systemctl enable kubelet && systemctl start kubelet
此时kubelet会启动不来，并且报一个/var/lib/kubelet/config.yaml文件找不到，而这个文件在kubeadm init的时候才会创建。
8、下载kubeadm相关的镜像
1)建立 init-config.yaml
cd /tempfile
touch init-config.yaml
cat < /tempfile/init-config.yaml
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
imageRepository: docker.io/dustise
kubernetesVersion: v1.14.0
networking:
podSubnet: "192.168.0.0/16"
EOF
2)下载镜像
kubeadm config images pull --config=/tempfile/init-config.yaml
一共会下载7个镜像，如果有报错下不来，应该是网络的原因，多下几次就好。
9、初始化Master节点（只在master上操作）
kubeadm init --config=init-config.yaml
一定要记录下token
[root@k8s-master01 ~]# kubeadm init --config=/tempfile/init-config.yaml
[init] Using Kubernetes version: v1.14.0
[preflight] Running pre-flight checks
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [k8s-master01 localhost] and IPs [192.168.1.51 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [k8s-master01 localhost] and IPs [192.168.1.51 127.0.0.1 ::1]
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [k8s-master01 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.1.51]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[apiclient] All control plane components are healthy after 47.541844 seconds
[upload-config] storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.14" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --experimental-upload-certs
[mark-control-plane] Marking the node k8s-master01 as control-plane by adding the label "node-role.kubernetes.io/master=''"
[mark-control-plane] Marking the node k8s-master01 as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: hn1v71.g1krss1rhia8wi6z
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] creating the "cluster-info" ConfigMap in the "kube-public" namespace
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.1.51:6443 --token hn1v71.g1krss1rhia8wi6z \
--discovery-token-ca-cert-hash sha256:f382265a741679c8356854920cd48f5a7306a1b05b751fe72463a8f5aa858fc5
查看kubelet状态systemctl status kubelet。此时启动了。
kubeadm init主要执行了以下操作：
-
List item
-
List item[init]：指定版本进行初始化操作
-
List item[preflight] ：初始化前的检查和下载所需要的Docker镜像文件
-
List item[kubelet-start]：生成kubelet的配置文件”/var/lib/kubelet/config.yaml”，没有这个文件kubelet无法启动，所以初始化之前的kubelet实际上启动失败。
-
List item[certificates]：生成Kubernetes使用的证书，存放在/etc/kubernetes/pki目录中。
-
List item[kubeconfig] ：生成 KubeConfig 文件，存放在/etc/kubernetes目录中，组件之间通信需要使用对应文件。
-
List item[control-plane]：使用/etc/kubernetes/manifest目录下的YAML文件，安装 Master 组件。
-
List item[etcd]：使用/etc/kubernetes/manifest/etcd.yaml安装Etcd服务。
-
List item[wait-control-plane]：等待control-plan部署的Master组件启动。
-
List item[apiclient]：检查Master组件服务状态。
-
List item[uploadconfig]：更新配置
-
List item[kubelet]：使用configMap配置kubelet。
-
List item[patchnode]：更新CNI信息到Node上，通过注释的方式记录。
-
List item[mark-control-plane]：为当前节点打标签，打了角色Master，和不可调度标签，这样默认就不会使用Master节点来运行Pod。
-
List item[bootstrap-token]：生成token记录下来，后边使用kubeadm join往集群中添加节点时会用到
-
List item[addons]：安装附加组件CoreDNS和kube-proxy
说明：无论是初始化失败或者集群已经完全搭建成功，你都可以直接执行kubeadm reset命令清理集群或节点，然后重新执行kubeadm init或kubeadm join相关操作即可。
10、配置kubectl命令（这步在master和node上一起做）
root用户执行以下命令
cat << EOF >> ~/.bashrc
export KUBECONFIG=/etc/kubernetes/admin.conf
EOF
source ~/.bashrc
root和普通用户执行以下命令（参考init时的输出结果）
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
等集群配置完成后，可以在所有master节点和node节点进行以上配置，以支持kubectl命令。针对node节点复制任意master节点/etc/kubernetes/admin.conf到本地。
11、在master上查看当前状态
此时可以看到master的状态为notready，这是由于未安装网络插件，coredns处于pending状态，node处于notready状态。
[root@k8s-master01 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master01 NotReady master 81s v1.14.1
[root@k8s-master01 ~]# kubectl -n kube-system get pod
NAME READY STATUS RESTARTS AGE
coredns-8686dcc4fd-cbrc5 0/1 Pending 0 64s
coredns-8686dcc4fd-wqpwr 0/1 Pending 0 64s
etcd-k8s-master01 1/1 Running 0 16s
kube-apiserver-k8s-master01 1/1 Running 0 13s
kube-controller-manager-k8s-master01 1/1 Running 0 25s
kube-proxy-4vwbb 1/1 Running 0 65s
kube-scheduler-k8s-master01 1/1 Running 0 4s
[root@k8s-master01 ~]# kubectl get cs
NAME STATUS MESSAGE ERROR
scheduler Healthy ok
controller-manager Healthy ok
etcd-0 Healthy {"health":"true"}
12、在master上安装网络插件
安装flannel网络插件：
由于kube-flannel.yml文件指定的镜像从coreos镜像仓库拉取，可能拉取失败，可以从dockerhub搜索相关镜像进行替换，另外可以看到yml文件中定义的网段地址段为10.244.0.0/16。
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
cat kube-flannel.yml | grep image
cat kube-flannel.yml | grep 10.244
sed -i 's#quay.io/coreos/flannel:v0.11.0-amd64#willdockerhub/flannel:v0.11.0-amd64#g' kube-flannel.yml
kubectl apply -f kube-flannel.yml
再次查看node和 Pod状态，全部为Running
[root@k8s-master01 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master01 Ready master 9m8s v1.14.1
[root@k8s-master01 ~]# kubectl -n kube-system get pod
NAME READY STATUS RESTARTS AGE
coredns-8686dcc4fd-cbrc5 1/1 Running 0 8m53s
coredns-8686dcc4fd-wqpwr 1/1 Running 0 8m53s
etcd-k8s-master01 1/1 Running 0 8m5s
kube-apiserver-k8s-master01 1/1 Running 0 8m2s
kube-controller-manager-k8s-master01 1/1 Running 0 8m14s
kube-flannel-ds-amd64-vtppf 1/1 Running 0 115s
kube-proxy-4vwbb 1/1 Running 0 8m54s
kube-scheduler-k8s-master01 1/1 Running 0 7m53s
13、加入master节点，此步骤在node上操作
kubeadm join 192.168.1.51:6443 --token hn1v71.g1krss1rhia8wi6z \
--discovery-token-ca-cert-hash sha256:f382265a741679c8356854920cd48f5a7306a1b05b751fe72463a8f5aa858fc5
可以看到node上的kubelet也已经启动了
systemctl status kubelet
14、在master上查看node状态
[sysadmin@k8s-master01 ~]$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master01 Ready master 132m v1.14.0
k8s-node01 Ready 120m v1.14.0
[sysadmin@k8s-master01 ~]$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-6897bd7b5-7vqfw 1/1 Running 0 133m
kube-system coredns-6897bd7b5-gk2xc 1/1 Running 0 133m
kube-system etcd-k8s-master01 1/1 Running 0 132m
kube-system kube-apiserver-k8s-master01 1/1 Running 0 132m
kube-system kube-controller-manager-k8s-master01 1/1 Running 0 132m
kube-system kube-flannel-ds-amd64-mm42f 1/1 Running 0 121m
kube-system kube-flannel-ds-amd64-szg27 1/1 Running 0 124m
kube-system kube-proxy-q7lzd 1/1 Running 0 121m
kube-system kube-proxy-zqfjw 1/1 Running 0 133m
kube-system kube-scheduler-k8s-master01 1/1 Running 0 132m