用kubeadm搭建单master k8s集群

准备工作

• 环境
  CentOS-7-1810
Kubernets 1.15.0
docker 18.09.8

换为aliyun源

备份/etc/yum.repos.d/，然后删除该目录下所有repo

• 下载centos基础yum源配置

curl -o CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo

• 下载docker的yum源配置

curl -o docker-ce.repo https://download.docker.com/linux/centos/docker-ce.repo

• 配置kubernetes的yum源

cat < /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
        http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

刷新yum源缓存

yum clean all 
yum makecache

安装docker，推荐18.06.2

• 安装好后设置daemon

# Setup daemon.
cat > /etc/docker/daemon.json <

• 打开net.bridge.bridge-nf-call-iptables net.ipv4.ip_forward net.bridge.bridge-nf-call-ip6tables

modprobe overlay
modprobe br_netfilter

# Setup required sysctl params, these persist across reboots.
cat > /etc/sysctl.d/99-kubernetes-cri.conf <

关闭SELinux

setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

关闭swap

vi /etc/fstab
注释swap分区
# /dev/mapper/centos-swap swap                    swap    defaults        0 0

#保存退出vi后执行
swapoff -a

• 将主机名加入/etc/hosts

echo "192.168.1.200  k8s-master" >> /etc/hosts

• 安装kubeadm

yum install -y kubeadm

• 启动kubelet systemctl enable kubelet，否则一会kubeadm init会报一个Warning

• kubeadm初始化

kubeadm init --pod-network-cidr=10.244.0.0/16 #10.244.0.0/16为node子网段，不可与物理机网段有重叠

• 处理faild to pull image错误

#从aliyun拉取镜像，然后更名为相应镜像名
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.16.2
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.16.2
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.16.2
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.16.2
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.1
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.3.15-0
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.6.2

# 更名
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.16.2 k8s.gcr.io/kube-apiserver:v1.16.2
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.16.2 k8s.gcr.io/kube-controller-manager:v1.16.2
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.16.2 k8s.gcr.io/kube-scheduler:v1.16.2
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.16.2 k8s.gcr.io/kube-proxy:v1.16.2
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.1 k8s.gcr.io/pause:3.1
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.3.15-0 k8s.gcr.io/etcd:3.3.15-0
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.6.2 k8s.gcr.io/coredns:1.6.2

# 然后再执行
kubeadm init --pod-network-cidr=10.244.0.0/16 #For flannel to work correctly, you must pass --pod-network-cidr=10.244.0.0/16 to kubeadm init

Your Kubernetes control-plane has initialized successfully!

• 然后根据提示

# To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.1.117:6443 --token sghchj.79ktfi1uf27l4832 \
    --discovery-token-ca-cert-hash sha256:9e10e5391fbfaa493ec058c180523c4b4c04aac0c5cd94fc674103404b85a39f

预定义yaml文件的位置/etc/kubernetes/manifests

安装addon

1. CoreDNS: : cncf项目，主要是用来做服务发现，目前已经取代kube-dns作为k8默认的服务发现组件
2. kube-proxy: 基于iptables来做的负载均衡，service会用到

• 查看已安装的组件

kubectl get pods -n kube-system
NAME                                 READY   STATUS    RESTARTS   AGE
coredns-5c98db65d4-4g9f7             0/1     Pending   0          31m
coredns-5c98db65d4-gkkb7             0/1     Pending   0          31m
etcd-k8s-master                      1/1     Running   0          30m
kube-apiserver-k8s-master            1/1     Running   0          30m
kube-controller-manager-k8s-master   1/1     Running   0          30m
kube-proxy-wn2r7                     1/1     Running   0          31m
kube-scheduler-k8s-master            1/1     Running   0          30m

coredns的两个pod都是pending状态，这是因为网络插件还没有安装

• 安装flannel网络插件

# ref https://github.com/coreos/flannel/blob/master/Documentation/kubernetes.md

#必须先安装flannel-rbac
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/k8s-manifests/kube-flannel-rbac.yml
#安装flannel
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/62e44c867a2846fefb68bd5f178daf4da3095ccb/Documentation/kube-flannel.yml

几分钟后可以看到pod都已经起来了

NAME                                 READY   STATUS    RESTARTS   AGE
coredns-5c98db65d4-98pt4             1/1     Running   0          6m
coredns-5c98db65d4-dpfwk             1/1     Running   0          6m
etcd-k8s-master                      1/1     Running   0          5m11s
kube-apiserver-k8s-master            1/1     Running   0          5m8s
kube-controller-manager-k8s-master   1/1     Running   0          5m8s
kube-flannel-ds-amd64-rttrk          1/1     Running   0          89s
kube-proxy-6c9g9                     1/1     Running   0          6m
kube-scheduler-k8s-master            1/1     Running   0          5m11s

再查看核心组件状态

kubectl get componentstatus
NAME                 STATUS    MESSAGE             ERROR
scheduler            Healthy   ok
controller-manager   Healthy   ok
etcd-0               Healthy   {"health":"true"}

node的状态是Ready，说明我们的master安装成功，至此大功告成！
默认的master节点是不能调度应用pod的，所以我们还需要给master节点打一个污点标记

kubectl taint nodes --all node-role.kubernetes.io/master-

安装DashBoard

k8s 1.16.0以上请kubectl create -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta8/aio/deploy/recommended.yaml

• DashBoard会拉取k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1这个镜像，然鹅被墙了
  于是我们只能

docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kubernetes-dashboard-amd64:v1.10.1
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kubernetes-dashboard-amd64:v1.10.1 k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1

然后

# ref https://github.com/kubernetes/dashboard/blob/master/README.md
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml
默认使用端口：8001
默认镜像下载地址k8s.gcr.io  #被墙

可以看到DashBoard已经起来了

kubectl get po -n kube-system
NAME                                    READY   STATUS    RESTARTS   AGE
coredns-5c98db65d4-98pt4                1/1     Running   0          28m
coredns-5c98db65d4-dpfwk                1/1     Running   0          28m
etcd-k8s-master                         1/1     Running   0          27m
kube-apiserver-k8s-master               1/1     Running   0          27m
kube-controller-manager-k8s-master      1/1     Running   0          27m
kube-flannel-ds-amd64-rttrk             1/1     Running   0          24m
kube-proxy-6c9g9                        1/1     Running   0          28m
kube-scheduler-k8s-master               1/1     Running   0          27m
kubernetes-dashboard-7d75c474bb-tzcw9   1/1     Running   0          2s

• 备用部署方法——换镜像换端口
• 拉取官方Dashboard的yaml配置文件

wget https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml

• 修改拉取镜像地址

sed -i 's/k8s.gcr.io/registry.cn-hangzhou.aliyuncs.com\/google_containers/g' kubernetes-dashboard.yaml  #aliyun服务器

• 修改访问端口

sed -i '/targetPort:/a\ \ \ \ \ \ nodePort: 30001\n\ \ type: NodePort' kubernetes-dashboard.yaml

• 部署本地Dashboard yaml配置

kubectl create -f kubernetes-dashboard.yaml

• 检查相关服务状态

kubectl get deployment kubernetes-dashboard -n kube-system

kubectl get pods -n kube-system -o wide

kubectl get services -n kube-system

netstat -ntlp|grep 30001

• 创建Dashboard用户和授权

kubectl create serviceaccount  dashboard-admin -n kube-system
kubectl create clusterrolebinding  dashboard-admin --clusterrole=cluster-admin --serviceaccount=kube-system:dashboard-admin

• 查看访问Dashboard的令牌

kubectl describe secrets -n kube-system $(kubectl -n kube-system get secret | awk '/dashboard-admin/{print $1}')

浏览器访问Dashboardhttps://192.168.1.200:30001

• 输入token

  
  
  image.png

错误处理

# 使用journalctl -xe查看日志有时会发现以下错误信息
summary.go:92] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get container info for "/system.slice/docker.service": unknown container "/system.slice/docker.service"

• 编辑/usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf (或/etc/systemd/system/kubelet.service.d/10-kubeadm.conf)

新增: Environment="KUBELET_MY_ARGS=--runtime-cgroups=/systemd/system.slice --kubelet-cgroups=/systemd/system.slice"

修改ExecStart: 在末尾追加 $KUBELET_MY_ARGS