vmware虚拟机Centos7上部署kubernetes1.18

1.安装虚拟机

使用vmware安装一个cenos7虚拟机，只需要安装一个，其他2个复制即可，不过要改MAC地址，要不然可能获取不到IP地址

2.系统配置

主机一个master01（192.168.11.69），一个node01（192.168.11.70）,一个node02（192.168.11.71）,以下的配置都需要在三个主机上分别执行

2.1.关闭交换分区

swapoff -a 

2.2.关闭防火墙

[root@master01 ~]# systemctl stop firewalld
[root@master01 ~]# systemctl disable firewalld

2.3.关闭selinux

 2.4.添加阿里源

[root@master01 ~]# rm -rfv /etc/yum.repos.d/*
[root@master01 ~]# curl -o /etc/yum.repos.d/CentOS-Base.repo ttp://mirrors.aliyun.com/repo/Centos-7.repo

 2.5.配置主机名

master01： 

[root@master01 ~]# hostnamectl set-hostname master01
[root@master01 ~]# more /etc/hostname  

node01: 

[root@master01 ~]# hostnamectl set-hostname node01
[root@master01 ~]# more /etc/hostname  

node02: 

[root@master01 ~]# hostnamectl set-hostname node02
[root@master01 ~]# more /etc/hostname  

 2.6.配置内核参数，将桥接的IPv4流量传递到iptables的链

[root@master01 ~]# cat > /etc/sysctl.d/k8s.conf <

注：k8s该网络需要设置内核参数bridge-nf-call-iptables=1，没有这个后面添加网络的时候会报错。

3.安装软件

 3.1.安装开发工具

[root@master01 ~]# yum install vim bash-completion net-tools gcc -y

 3.2.安装docker-ce

[root@master01 ~]# yum install -y yum-utils device-mapper-persistent-data lvm2
[root@master01 ~]# yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
[root@master01 ~]# yum -y install docker-ce

 

注：yum-config-manager命令配置aliyun源，但是这个命令来源于yum-utils，所以需要先安装yum-utils

 3.3.安装完docker后添加aliyun的docker仓库加速器

[root@master01 ~]# mkdir -p /etc/docker
[root@master01 ~]# tee /etc/docker/daemon.json <<-'EOF'
{
  "registry-mirrors": ["https://fl791z1h.mirror.aliyuncs.com"]
}
EOF

 

[root@master01 ~]# systemctl daemon-reload
[root@master01 ~]# systemctl restart docker

注：tee /etc/docker/daemon.json <<-'EOF’后面的 <<-'EOF’是没有空格的，如图我第一次有空格输入EOF就没有结束

3.4.安装kubectl、kubelet、kubeadm 

先添加阿里kubernetes源

[root@master01 ~]# cat < /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

安装

[root@master01 ~]# yum -y install kubectl-1.18.0 kubelet-1.18.0 kubeadm-1.18.0
[root@master01 ~]# systemctl enable kubelet

注： yum install kubectl kubelet kubeadm，这样写会默认安装最新版,但是和我后面执行的kubeadm init --kubernetes-version=1.18.0版本不一致，会报错，所以我这里yum后面指定了版本。

4.初始化k8s集群(master节点) 

 

这一步开始区分master、ndoe节点执行的命令了，上面的步骤master、node都是一样

[root@master01 ~]# kubeadm init --kubernetes-version=1.18.0  \
--apiserver-advertise-address=192.168.11.69   \
--image-repository registry.aliyuncs.com/google_containers  \
--service-cidr=10.10.0.0/16 --pod-network-cidr=10.122.0.0/16

执行完后出现如下信息：

W1125 22:47:32.274048   47607 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
[init] Using Kubernetes version: v1.18.0
[preflight] Running pre-flight checks
        [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [localhost.localdomain kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.10.0.1 192.168.137.110]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost.localdomain localhost] and IPs [192.168.137.110 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost.localdomain localhost] and IPs [192.168.137.110 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
W1125 22:47:35.941950   47607 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"
[control-plane] Creating static Pod manifest for "kube-scheduler"
W1125 22:47:35.943106   47607 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 21.504047 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.18" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node localhost.localdomain as control-plane by adding the label "node-role.kubernetes.io/master=''"
[mark-control-plane] Marking the node localhost.localdomain as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: o11ulw.ovtj9vov7ob34hs7
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.11.69:6443 --token o11ulw.ovtj9vov7ob34hs7 \
    --discovery-token-ca-cert-hash sha256:38a93626ac4e86155583e7ef9b32cb13739d5f5bc3da2b4ed7e74aec8112bea7

记录后面的kubeadm join这段内容，此内容需要在其它节点加入Kubernetes集群时执行。

kubeadm join 192.168.11.69:6443 --token o11ulw.ovtj9vov7ob34hs7 \
    --discovery-token-ca-cert-hash sha256:38a93626ac4e86155583e7ef9b32cb13739d5f5bc3da2b4ed7e74aec8112bea7

创建kubectl

[root@master01 ~]# mkdir -p $HOME/.kube
[root@master01 ~]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@master01 ~]# sudo chown $(id -u):$(id -g) $HOME/.kube/config

注：
1.不配置$HOME/.kube/config的话，kubectl命令不可用，
2.node节点写法有点不一样，node节点的这行为：sudo cp -i /etc/kubernetes/kubelet.conf.conf $HOME/.kube/config

5.安装calico网络（master节点）

[root@master01 ~]# kubectl apply -f https://docs.projectcalico.org/v3.18/manifests/calico.yaml

注：现在的版本是3.23，需要指定版本否则会报错，版本对应关系可以看这里：https://docs.tigera.io/archive/v3.18/getting-started/kubernetes/requirements
安装calico网络网络后过一会再输入kubectl get node，可以看到节点的STATUS由NotReady变为Ready

[root@master01 ~]# kubectl get node
NAME       STATUS   ROLES    AGE    VERSION
master01   Ready    master   2d1h   v1.18.0
node01     Ready       2d1h   v1.18.0
node02     Ready       2d1h   v1.18.0

6.node节点加入集群（两个node节点都要执行）

[root@node01 ~]# kubeadm join 192.168.11.69:6443 --token o11ulw.ovtj9vov7ob34hs7     --discovery-token-ca-cert-hash sha256:38a93626ac4e86155583e7ef9b32cb13739d5f5bc3da2b4ed7e74aec8112bea7

 注：
1.kubeadm init后得到的token有效期为24小时，过期后需要重新创建token,执行：kubeadm token create获取新token
2.kubeadm token list 查看token列表，

创建kubectl

[root@master01 ~]# mkdir -p $HOME/.kube
[root@master01 ~]# sudo cp -i /etc/kubernetes/kubelet.conf $HOME/.kube/config
[root@master01 ~]# sudo chown $(id -u):$(id -g) $HOME/.kube/config

到了这步，把所有的node节点加入mater节点后，k8s的环境已经安装完成了

[root@master01 ~]# kubectl get node
NAME       STATUS   ROLES    AGE    VERSION
master01   Ready    master   2d1h   v1.18.0
node01     Ready       2d1h   v1.18.0
node02     Ready       2d1h   v1.18.0

7.安装kubernetes-dashboard(master节点）

 

Dashboard是可视化插件，它可以给用户提供一个可视化的 Web 界面来查看当前集群的各种信息。用户可以用 Kubernetes Dashboard 部署容器化的应用、监控应用的状态、执行故障排查任务以及管理 Kubernetes 各种资源。

 官方部署dashboard的服务没使用nodeport，将yaml文件下载到本地，在service里添加nodeport

[root@master01 ~]# wget  https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-rc7/aio/deploy/recommended.yaml
[root@master01 ~]# vim recommended.yaml
kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  type: NodePort
  ports:
    - port: 443
      targetPort: 8443
      nodePort: 30000
  selector:
    k8s-app: kubernetes-dashboard

[root@master01 ~]# kubectl create -f recommended.yaml

 

执行

kubectl create -f recommended.yaml

 如果出错，例如配置错误可以执行一下命令恢复，然后支持create或者apply

kubectl delete -f recommended.yaml

执行kubectl create -f recommended.yaml命令后，再执行下面这行，可以看到dashboard已启动 

[root@master01 ~]# kubectl get svc -n kubernetes-dashboard
NAME                        TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)         AGE
dashboard-metrics-scraper   ClusterIP   10.10.247.69           8000/TCP        46h
kubernetes-dashboard        NodePort    10.10.120.13           443:30001/TCP   46h

 在浏览器输入我们的这台机器的ip+端口，进入登录页面

 

在空白地方输入thisisunsafe就可以了

获取token

 

kubectl -n kubernetes-dashboard get secret
kubectl describe secrets -n kubernetes-dashboard kubernetes-dashboard-token-tljmr  | grep token | awk 'NR==3{print $2}'

 把token粘贴到登录页面上的输入token框，点击登录

 

但是都是空白的，右上角报错

 报错如下：

 

 原因：serviceaccount的问题,k8sdashboard出厂的serviceaccount权限太低，需要配置一个admin用户

[root@master01 ~]# cat >user.yml 
apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin
  namespace: kubernetes-dashboard

创建集群RoleBinding.yml

[root@master01 ~]# cat > ClusterRoleBinding.yml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin
  namespace: kubernetes-dashboard

 执行结果：

[root@master01 ~]# kubectl apply -f user.yml 
serviceaccount/admin created
[root@master01 ~]# kubectl apply -f ClusterRoleBinding.yml 
clusterrolebinding.rbac.authorization.k8s.io/admin created

[root@master01 ~]# kubectl get -f user.yml 
NAME    SECRETS   AGE
admin   1         47h
[root@master01 ~]#  kubectl get -f ClusterRoleBinding.yml
NAME    ROLE                        AGE
admin   ClusterRole/cluster-admin   47h

[root@master01 ~]# kubectl get serviceaccount -n kubernetes-dashboard
NAME                   SECRETS   AGE
admin                  1         47h
default                1         47h
kubernetes-dashboard   1         47h 

 重现获取token登录：

[root@master01 ~]# kubectl -n kubernetes-dashboard get secret
NAME                               TYPE                                  DATA   AGE
admin-token-wmr7b                  kubernetes.io/service-account-token   3      47h
default-token-srztq                kubernetes.io/service-account-token   3      47h
kubernetes-dashboard-certs         Opaque                                0      47h
kubernetes-dashboard-csrf          Opaque                                1      47h
kubernetes-dashboard-key-holder    Opaque                                2      47h
kubernetes-dashboard-token-js42w   kubernetes.io/service-account-token   3      47h
[root@master01 ~]# kubectl describe secrets -n kubernetes-dashboard admin-token-wmr7b
Name:         admin-token-wmr7b
Namespace:    kubernetes-dashboard
Labels:       
Annotations:  kubernetes.io/service-account.name: admin
              kubernetes.io/service-account.uid: 6b3a1d5c-8529-4381-bce9-5504d01d2983

Type:  kubernetes.io/service-account-token

Data
====
ca.crt:     1025 bytes
namespace:  20 bytes
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IklCY3FzM1A2N0Z4dFMzQmVSa0NzTEJ4VVkyVFI1T3lqWnBKV1EzVWh5eWsifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi10b2tlbi13bXI3YiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJhZG1pbiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjZiM2ExZDVjLTg1MjktNDM4MS1iY2U5LTU1MDRkMDFkMjk4MyIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlcm5ldGVzLWRhc2hib2FyZDphZG1pbiJ9.w4ATRjr624fMAGwRlz6n7VlxphUU9fSM2rPK71uJK-KO1fWKN1LyDmf2ZGHv3G-jlWpXEINSGmXwjiPl9TXLkTriX3rwOJ8jZUsziBiNig1dx8rA-wgqLTwGmsVILBtvwkMvTNhqTl_CVQj3P8d98l8S_eXTL_ZcykaWLyaltGIwGUYZLnyyL3fyQonJudgc2NmAU2jOZnUaWqw1zsElE_XJKzkVgE16ETWxwiOmSVClo771zdpVcWkzk5FRk2fN4R4rHoRoC9kkCMvkHBtz7bT28m-fYGYrANCaJUiXwLaok1UQIj9J9Dt6Ja2qksknkGIQ5kezefBG-zgUoP5dJQ
[root@master01 ~]# 

 

 完结