1. kubernetes 的安装与部署

环境信息: centos7.9 + kubernetes 1.23.8 + docker 20.10.17 + virtualBox 6.1
文章编写时间: 2022-06-30
部署方式: kubeadm
组件: 网络组件calico、dashboard组件

一、前置工作与注意事项

这里我们的centos使用的是 centos7.9, 不同版本的系统对k8s影响较大，具体看实际情况而定。有的还需要更新系统内核。
我们先准备了3台虚拟机，配置好网络(映射好ssh端口)。虚拟机情况如下：
```
k8s_master-1  192.168.56.105
k8s_slave-2   192.168.56.106
k8s_slave-3   192.168.56.107
```
hosts配置(每台机器都需要设置)
```
vim /etc/hosts

192.168.56.105 k8s-master01
192.168.56.106 k8s-slave02
192.168.56.107 k8s-slave03
```
hostname配置(每台机器都需要配置，这里我们以192.168.56.105 为例，我们需要设置hostname为 "k8s_master-1" ，与hosts 相匹配)

如果不配置hostname 默认会配置为localhost.localdomain，k8s 运行时会报错Error getting node" err="node \"localhost.localdomain\" not found
```
# 设置当前机器的hostname
hostnamectl set-hostname k8s-master01
# 查看当前机器hostname
hostname
```
系统配置要求：2c 2g 20g 以上,， cpu 至少为2核，否则k8s初始化无法成功。建议master节点内存给4g
k8s安装有多种方式:
1. 使用minikube安装单节点集群，用于测试
2. 采用工具kubeadm -- 我们使用的这种方式（开发环境，机器比较少(几十台以下)）
3. 使用kubespray, google官方提供的工具
4. 全手动: 二进制安装(运维)
5. 全自动安装: rancher、kubesphere (大型生产环境，百台，万台机器)
k8s health会依赖一些端口，为了不出现网络问题，我们在虚拟机(master)中开放以下端口：
- 6443 主要
- 2379
- 2380
- kubeadm 帮助我们安装的ca 证书时限是一年，所以不推荐正式环境使用，或需要手动配置ca证书。

二、安装

1. 初始准备

以下为实际安装步骤与流程:

# 基础依赖包安装
 yum -y install wget vim net-tools ntpdate  bash-completion
 
# 设置当前机器的hostname
hostnamectl set-hostname k8s-master01
# 查看当前机器hostname
hostname

0.系统时钟同步

# 向阿里云服务器同步时间
ntpdate time1.aliyun.com
# 删除本地时间并设置时区为上海
rm -rf /etc/localtime
ln -s /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
# 查看时间
date -R || date

1.关闭防火墙、selinux

 systemctl stop firewalld
 systemctl disable firewalld
 sed -i 's/enforcing/disabled/' /etc/selinux/config
 setenforce 0

3.关闭swap

# 临时关闭Swap
swapoff -a
# 修改 /etc/fstab 删除或者注释掉swap的挂载，可永久关闭swap
sed -i '/swap/s/^/#/' /etc/fstab
# 修改完后我们检测以下，看最后一行swap 都是0 就成功了
free -m
#----------------start----------------------
              total        used        free      shared  buff/cache   available
Mem:           1837         721          69          10        1046         944
Swap:             0           0           0
#-----------------end---------------------

4.网桥过滤

# 网桥过滤
vim /etc/sysctl.conf

net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-arptables = 1
net.ipv4.ip_forward=1
net.ipv4.ip_forward_use_pmtu = 0

# 生效命令
sysctl --system
# 查看效果
sysctl -a|grep "ip_forward"

5.开启ipvs（kubernetes1.8版本开始，新增了kube-proxy对ipvs的支持，性能和追踪问题比iptable强）------ 此步骤为选填项，如果不执行那么默认使用iptables

# 安装IPVS
yum -y install ipset ipvsdm
# 编译ipvs.modules文件
vi /etc/sysconfig/modules/ipvs.modules
# 文件内容如下
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
# 赋予权限并执行
chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules &&lsmod | grep -e ip_vs -e nf_conntrack_ipv4
# 重启电脑，检查是否生效
reboot
lsmod | grep ip_vs_rr

修改hosts文件，添加解析

vim /etc/hosts

192.168.56.105 k8s-master01
192.168.56.106 k8s-slave02
192.168.56.107 k8s-slave03

2. docker安装

docker 换源安装

# 安装yum utils
yum install -y yum-utils
# yum docker-ce config 换源
yum-config-manager \
    --add-repo \
    https://download.docker.com/linux/centos/docker-ce.repo
# 安装docker
yum -y install docker-ce docker-ce-cli containerd.io
# 启动docker， enable 为必须，k8s会检测docker.service
systemctl enable docker && systemctl start docker

docker配置镜像加速

# 创建docker目录
mkdir -p /etc/docker
# 设置镜像源, exec-opts必须指定否则k8s启动报错（cgroup、systemd）
tee /etc/docker/daemon.json <<-'EOF'
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "registry-mirrors": ["https://fl791z1h.mirror.aliyuncs.com"]
}
EOF
# 重启docke并生效镜像加速
systemctl daemon-reload && systemctl restart docker

3. k8s安装

配置kubernetes源

cat < /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

注意：阿里源并未与官网同步gpg（由于官网未开放同步方式, 可能会有索引gpg检查失败的情况，这时请用如下命令安装）

安装kubernets,最好指定版本，否则会使用最新版本(k8s 每个版本的变化都比较大，这里我们的k8s使用1.23.8 版本)

# 检测可用的k8s版本(--nogpgcheck 忽略gpg检测)
yum list --nogpgcheck  --showduplicates kubeadm --disableexcludes=kubernetes
# 找到我们想要安装的版本，并安装--------------这里我们用1.23.8版本，最新版目前是1.24.0 版本安装启用了docker 会有一些问题。
# 安装kubelet、kubeadm、kubectl 组件--- 这里要注意，docker 版本和 k8s版本有关系，尽量使用支持区间的版本
# yum install --nogpgcheck kubelet-1.23.8 kubeadm-1.23.8 kubectl-1.23.8
yum -y install --nogpgcheck kubelet-1.23.8 kubeadm-1.23.8 kubectl-1.23.8

安装完成后我们检查一下

# 检查kubectl version
kubectl version
##########show start############
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.23.8", GitCommit:"5575935422cc1cf5169dfc8847cb587aa47bac5a", GitTreeState:"clean", BuildDate:"2021-06-16T13:00:45Z", GoVersion:"go1.15.13", Compiler:"gc", Platform:"linux/amd64"}
The connection to the server localhost:8080 was refused - did you specify the right host or port?
###########show end###########
# 检查kubeadm版本
kubeadm version
##########show start############
kubeadm version: &version.Info{Major:"1", Minor:"20", GitVersion:"v1.23.8", GitCommit:"5575935422cc1cf5169dfc8847cb587aa47bac5a", GitTreeState:"clean", BuildDate:"2021-06-16T12:58:46Z", GoVersion:"go1.15.13", Compiler:"gc", Platform:"linux/amd64"}
##########show end############

启动k8s服务

# 启动k8s服务
systemctl enable kubelet && systemctl start kubelet
# 查看服务状态
systemctl status kubelet

# 如果不指定版本初始化那么会使用最新的k8s，有可能存在的报错信息如下，需要先手动设置下
# 运行初始化后有可能会报错:
#########错误信息--start###########
[init] Using Kubernetes version: v1.24.2
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
        [ERROR CRI]: container runtime is not running: output: E0627 16:44:11.772277   16359 remote_runtime.go:925] "Status from runtime service failed" err="rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
time="2022-06-27T16:44:11+08:00" level=fatal msg="getting status of runtime: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
#########错误信息--end###########
# 解决方案：
vim /etc/containerd/config.toml
# 从disabled_plugins 数组值中删除 "cri"属性
# 重启容器
systemctl restart containerd

k8s master 主节点初始化（仅master节点执行-- 这里考虑的是单master，多slave）

# 初始化
kubeadm init  \
--image-repository registry.aliyuncs.com/google_containers  \
--apiserver-advertise-address=192.168.56.105  \
--service-cidr=10.222.0.0/16 \
--pod-network-cidr=10.244.0.0/16

# 初始化过程比较长，需要下载一些资源

#---------------打印信息 start---------------------
I0628 15:14:20.293469    5655 version.go:255] remote version is much newer: v1.24.2; falling back to: stable-1.23
[init] Using Kubernetes version: v1.23.8
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost.localdomain] and IPs [10.222.0.1 192.168.56.105]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost localhost.localdomain] and IPs [192.168.56.105 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost localhost.localdomain] and IPs [192.168.56.105 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 4.502756 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.23" in namespace kube-system with the configuration for the kubelets in the cluster
NOTE: The "kubelet-config-1.23" naming of the kubelet ConfigMap is deprecated. Once the UnversionedKubeletConfigMap feature gate graduates to Beta the default name will become just "kubelet-config". Kubeadm upgrade will handle this transition transparently.
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node localhost.localdomain as control-plane by adding the labels: [node-role.kubernetes.io/master(deprecated) node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node localhost.localdomain as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: 4yipfl.er9r8aqnq0hpd8a4
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.56.105:6443 --token 4yipfl.er9r8aqnq0hpd8a4 \
        --discovery-token-ca-cert-hash sha256:afa76da3ced528e667374693fb4b0edd160530c251471ae11ece13c65d3d162a

#---------------打印信息 end---------------------

初始化参数解析

参数名	示例值	含义
--kubernetes-version	v1.23.8	版本
--apiserver-advertise-address	192.168.56.105	当前机器节点ip
--image-repository	registry.aliyuncs.com/google_containers	镜像仓库
--service-cidr	10.222.0.0/16	service 网段
--pod-network-cidr	10.244.0.0/16	k8s内部pod节点直接网段，不能和--service-cide相同

至此，kubeadm init(master 主节点)安装完成。

还需要进行一些收尾工作，根据kubeadm init log 提示，执行以下命令

mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config

然后，我们就可以进行k8s的节点查询

# 查询节点
kubectl get nodes

#------------展示信息 start--------------
NAME           STATUS     ROLES                  AGE     VERSION
k8s-master01   NotReady   control-plane,master   6m21s   v1.23.8
#------------展示信息 end----------------

此时， STATUS 是 NotReady状态，因为网络组件还未安装,Pod之间还不能通讯

查看各命名空间下的Pod信息

kubectl get pods --all-namespaces

#------------展示信息 start--------------

NAMESPACE     NAME                                   READY   STATUS    RESTARTS   AGE
kube-system   coredns-6d8c4cb4d-5bjmr                0/1     Pending   0          12m
kube-system   coredns-6d8c4cb4d-7w72l                0/1     Pending   0          12m
kube-system   etcd-k8s-master01                      1/1     Running   0          12m
kube-system   kube-apiserver-k8s-master01            1/1     Running   0          12m
kube-system   kube-controller-manager-k8s-master01   1/1     Running   0          12m
kube-system   kube-proxy-rcsfg                       1/1     Running   0          12m
kube-system   kube-scheduler-k8s-master01            1/1     Running   0          12m
#------------展示信息 end----------------

可以看到NDS解析服务coredns的pod还处于Pending状态未运行，也是因为网络组件还没安装

三、网络插件的安装

下面我们进行网络组件的安装

1. 常用网络插件

这里只简单说明下，推荐使用calico。

flannel 和 calico 是常用的网络插件。

calico 的性能更好，使用场景更广一些。

flannel 没有网络策略，不能控制pod的访问。

这里我们用calico插件

2. 插件安装

# calico插件安装
kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml

# 插件安装过程较慢，请耐心等待
# 安装后我们查看pod状态,直到 所有 STATUS 为 Running 才启动成功
kubectl get pod --all-namespaces

#----------显示如下 start-------------
NAMESPACE     NAME                                       READY   STATUS    RESTARTS   AGE
kube-system   calico-kube-controllers-7bc6547ffb-2bnbh   1/1     Running   0          5m57s
kube-system   calico-node-rnhcv                          1/1     Running   0          5m57s
kube-system   coredns-6d8c4cb4d-5bjmr                    1/1     Running   0          90m
kube-system   coredns-6d8c4cb4d-7w72l                    1/1     Running   0          90m
kube-system   etcd-k8s-master01                          1/1     Running   0          91m
kube-system   kube-apiserver-k8s-master01                1/1     Running   0          91m
kube-system   kube-controller-manager-k8s-master01       1/1     Running   0          91m
kube-system   kube-proxy-rcsfg                           1/1     Running   0          90m
kube-system   kube-scheduler-k8s-master01                1/1     Running   0          91m
#----------显示如下 end---------------

# 查看k8s node 状态
kubectl get nodes

#----------显示如下 start-------------
k8s-master01   Ready    control-plane,master   99m   v1.23.8
#----------显示如下 end---------------

四、安装dashboard

1.注意事项

dashboard 在github上开源地址:

dashboard 的版本和 k8s的版本有关系，因为每个k8s版本改动较大，所以在选择dashboard时尽量选择兼容的版本，否则某些功能有可能使用异常。

版本的选择方式：

github 点击releases
查看dashboard版本的兼容情况，并选择对应版本

image.png

3.找到支持的兼容版本，复制安装语句并执行

image.png

2.安装

# 安装dashboard
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.5.1/aio/deploy/recommended.yaml

#----------显示如下 start-------------
namespace/kubernetes-dashboard configured
serviceaccount/kubernetes-dashboard created
service/kubernetes-dashboard created
secret/kubernetes-dashboard-certs created
secret/kubernetes-dashboard-csrf created
secret/kubernetes-dashboard-key-holder created
configmap/kubernetes-dashboard-settings created
role.rbac.authorization.k8s.io/kubernetes-dashboard created
clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard created
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
deployment.apps/kubernetes-dashboard created
service/dashboard-metrics-scraper created
deployment.apps/dashboard-metrics-scraper created
#----------显示如下 end---------------

# 配置dashboard访问端口等信息
vim k8s-dashboard.yaml

#----------编辑内容 start------------- 
kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  type: NodePort
  ports:
    - port: 443
      targetPort: 8443
      # 对外暴露的端口（端口范围 30000~32767）
      nodePort: 30443
  selector:
    k8s-app: kubernetes-dashboard

#----------编辑内容 end--------------- 

# 运行yaml
kubectl apply -f k8s-dashboard.yaml

#----------显示如下 start-------------
service/kubernetes-dashboard created
#----------显示如下 end---------------

# 这样我们的dashboard服务就部署完毕了，下面我们验证下
# 查看pod，我们会看到两个pod 的status 都是`Running`
kubectl get pods -n kubernetes-dashboard

#----------显示如下 start-------------
NAME                                         READY   STATUS    RESTARTS   AGE
dashboard-metrics-scraper-799d786dbf-j85zw   1/1     Running   0          18m
kubernetes-dashboard-fb8648fd9-qcrzk         1/1     Running   0          18m
#----------显示如下 end---------------

# 查看服务
kubectl get svc -n kubernetes-dashboard -o wide
#----------显示如下 start-------------
NAME                        TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)         AGE   SELECTOR
dashboard-metrics-scraper   ClusterIP   10.222.211.134           8000/TCP        19m   k8s-app=dashboard-metrics-scraper
kubernetes-dashboard        NodePort    10.222.156.236           443:30443/TCP   19m   k8s-app=kubernetes-dashboard
#----------显示如下 end---------------

上面我们说到dashboard 开放的端口是30443，我们的虚拟机也需要对外暴露该端口

image.png

设置好后，我们访问https://192.168.56.105:30443（注意：这里一定要用https）

image.png

3.权限配置与登录验证

上面我们已经完成了dashboard的安装，但是我们发现登录的时候需要使用Token,下边我们就说下Token的生成和配置

# 为了规范化处理，我们制定配置文件路径
cd /opt/kube-dashboard/conf
# 创建rbac配置文件
vim admin-user-dashboard.yaml

#----------编辑内容 start------------- 
apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard
 
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kubernetes-dashboard
 
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
  name: cluster-view
rules:
- apiGroups:
  - '*'
  resources:
  - '*'
  verbs:
  - get
  - list
  - watch
- nonResourceURLs:
  - '*'
  verbs:
  - get
  - list
  - watch
 
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: view-user
  namespace: kubernetes-dashboard
 
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: view-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-view
subjects:
- kind: ServiceAccount
  name: view-user
  namespace: kubernetes-dashboard
#----------编辑内容 end--------------- 

# 运行权限配置
kubectl apply -f admin-user-dashboard.yaml

# 生成登录token
# admin-user 为角色名称
kubectl describe secret admin-user -n kubernetes-dashboard
# 或者(还是推荐上边的语句，比较简单)
# kubectl -n kubernetes-dashboard get secret $(kubectl -n kubernetes-dashboard get sa/admin-user -o jsonpath="{.secrets[0].name}") -o go-template="{{.data.token | base64decode}}"

#展示内容如下，我们复制token部分信息即可

#----------------------------------------------
Name:         admin-user-token-2gt2w
Namespace:    kubernetes-dashboard
Labels:       
Annotations:  kubernetes.io/service-account.name: admin-user
              kubernetes.io/service-account.uid: ce38f197-f395-45ed-9385-104707df07c7

Type:  kubernetes.io/service-account-token

Data
====
namespace:  20 bytes
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6ImgzM0taa283elhJbjFvc2NFNTJULXVDTGNURjZJaV9zSzV0X3U1VkgwUFEifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi11c2VyLXRva2VuLTJndDJ3Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJjZTM4ZjE5Ny1mMzk1LTQ1ZWQtOTM4NS0xMDQ3MDdkZjA3YzciLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZXJuZXRlcy1kYXNoYm9hcmQ6YWRtaW4tdXNlciJ9.RoYrf2VCgGn8RiHVgBvMS7El4DWa6XmAT_Prrjs_Kk2nOFOjG2z3i4I_1Db9S6Jq0ZC-L2lCeGDQSdMmOBW1eYMNw6-vSwteKR_Un7GhshPLK4AJML3CK8uHsgYnhM64EinyTcdbBj9ade6OdJ3ypFi_Dw_oms4CUnuD57zLynZnh_JGMj-HJEMmtjBDV_FE-yJUn7_Y626e5Uw92p_xcW9up68TPEMuOSTedlxHJ61jpGf0H8ZGdinslvgpEbp7jUJeXoU_caLHhKGc28pQzJgjtHkatHJS7HmYdcPmSSON-2HZztmNlNfHI0luEfEg2KCAU3hxQeDKMw89jye1eg
ca.crt:     1099 bytes

#----------------------------------------------

下面我们复制登录token，登录dashboard，显示如下页面

image.png

五、子节点加入

子节点需要执行 一、二两点操作，注意二点我们除了kubeadm init操作外其他的都执行

子节点加入的命令在 step3-》k8s安装-》kubeadm init 输出的日志中我们可以找到。

kubeadm join 192.168.56.105:6443 --token 4yipfl.er9r8aqnq0hpd8a4 \
        --discovery-token-ca-cert-hash sha256:afa76da3ced528e667374693fb4b0edd160530c251471ae11ece13c65d3d162a

下面我们简单说下子节点服务器的操作流程:

# 基础依赖包安装
yum -y install wget vim net-tools ntpdate bash-completion
# 修改当前机器名 
hostnamectl set-hostname k8s-slave02
# 或 hostnamectl set-hostname k8s-slave03

# 修改hosts文件
vim /etc/hosts

192.168.56.105 k8s-master01
192.168.56.106 k8s-slave02
192.168.56.107 k8s-slave03

# 系统时钟同步与时区配置
ntpdate time1.aliyun.com
rm -rf /etc/localtime
ln -s /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
date -R || date

# 关闭防火墙、selinux
systemctl stop firewalld
systemctl disable firewalld
sed -i 's/enforcing/disabled/' /etc/selinux/config
setenforce 0

# 关闭swap
swapoff -a
sed -i '/swap/s/^/#/' /etc/fstab
free -m

# 网桥过滤
vim /etc/sysctl.conf

net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-arptables = 1
net.ipv4.ip_forward=1
net.ipv4.ip_forward_use_pmtu = 0

# 生效命令 与 查看
sysctl --system
sysctl -a|grep "ip_forward"

# docker安装
yum install -y yum-utils
yum-config-manager \
--add-repo \
https://download.docker.com/linux/centos/docker-ce.repo
yum -y install docker-ce docker-ce-cli containerd.io
systemctl enable docker && systemctl start docker

# docker 镜像加速 与 cgroup配置
mkdir -p /etc/docker

tee /etc/docker/daemon.json <<-'EOF'
{
"exec-opts": ["native.cgroupdriver=systemd"],
"registry-mirrors": ["https://fl791z1h.mirror.aliyuncs.com"]
}
EOF

systemctl daemon-reload && systemctl restart docker

# k8s安装
cat < /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

yum -y install --nogpgcheck kubelet-1.23.8 kubeadm-1.23.8 kubectl-1.23.8

# 启动k8s服务
systemctl enable kubelet && systemctl start kubelet

# join k8s网络
kubeadm join 192.168.56.105:6443 --token 4yipfl.er9r8aqnq0hpd8a4 \
        --discovery-token-ca-cert-hash sha256:afa76da3ced528e667374693fb4b0edd160530c251471ae11ece13c65d3d162a
# 注意： token 有效时间是24小时，如果过来24小时那么需要刷新token
# 在master 主节点上运行命令，刷新 token
kubeadm token create --print-join-command
# 得到以下结果
kubeadm join 192.168.56.105:6443 --token o26r8i.zg2t9ade0tyuh4tp --discovery-token-ca-cert-hash sha256:de2d35d81f3740e93aca8a461713ca4ab1fcb9e7e881dc0f8836dd06d8a40229
# 在slave 节点上运行 上述 `kubeadm join` 语句
# 运行成功，显示如下:

#------------运行成功 start-----------------
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
#------------运行成功 end-------------------

# 在主节点上 查看所有nodes
kubectl get nodes


NAME           STATUS     ROLES                  AGE   VERSION
k8s-master01   Ready      control-plane,master   18h   v1.23.8
k8s-slave02    NotReady                    46s   v1.23.8

# 这个时候我们看到 `k8s-slave02` 已经加入到 k8s集群中了，但是`STATUS`还是`NotReady`
# 此步骤比较耗时，我们多等一会，直到 `NotReady` 变为 `Ready`

NAME           STATUS   ROLES                  AGE     VERSION
k8s-master01   Ready    control-plane,master   18h     v1.23.8
k8s-slave02    Ready                     3m16s   v1.23.8

# 至此，子节点加入成功
# 为了稳妥起见，我们在 master 上发现子节点 状态由 `NotReady` 变成 `Ready` 后，我们对子节点k8s 进行重启
systemctl restart kubelet

子节点上我们尝试运行 k8s命令

# 查看节点信息
kubectl get nodes
# 展示信息
The connection to the server localhost:8080 was refused - did you specify the right host or port?

我们发现：子节点无法运行kubectl命令

原因：kubectl命令需要使用 kubernetes-admin 来运行

解决办法: 我们将master节点的 admin.conf 复制到子节点上

# master节点上操作：从master节点 复制到 子节点
scp /etc/kubernetes/admin.conf [email protected]:/etc/kubernetes/
# scp /etc/kubernetes/admin.conf [email protected]:/etc/kubernetes/

# node节点上操作：配置环境变量
echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> ~/.bash_profile
source ~/.bash_profile

# 而后我们在子节点上运行 命令
kubectl get nodes

NAME           STATUS   ROLES                  AGE   VERSION
k8s-master01   Ready    control-plane,master   19h   v1.23.8
k8s-slave02    Ready                     49m   v1.23.8
k8s-slave03    Ready                     14m   v1.23.8

六、常用信息

1. 常用命令

# 查看k8s 运行日志命令, 这个比较有用，在k8s 启动、kubeadm init、kubeadm join 阶段可以辅助分析问题。
journalctl -xefu kubelet 
# 查看k8s驱动
systemctl show --property=Environment kubelet |cat
# 重启k8s
systemctl restart kubelet
# 启动k8s
systemctl start kubelet
# 停止k8s
systemctl stop kubelet
# 开机自启k8s
systemctl enable kubelet

# dashboard 获取token
kubectl describe secret admin-user -n kubernetes-dashboard

# kubeadm 重置， 有些时候我们在使用kubeadm init 命令时会报错，我们根据错误提示修复问题后需要重新进行 init 操作，因此需要进行reset重置
kubeadm reset

2. 环境信息

# k8s 安装目录
/etc/kubernetes/

总用量 32
-rw-------. 1 root root 5642 6月  28 15:19 admin.conf
-rw-------. 1 root root 5674 6月  28 15:19 controller-manager.conf
-rw-------. 1 root root 1986 6月  28 15:19 kubelet.conf
drwxr-xr-x. 2 root root  113 6月  28 15:19 manifests
drwxr-xr-x. 3 root root 4096 6月  28 15:19 pki
-rw-------. 1 root root 5618 6月  28 15:19 scheduler.conf

# 组件配置文件目录
/etc/kubernetes/manifests/

总用量 16
-rw-------. 1 root root 2310 6月  28 15:19 etcd.yaml
-rw-------. 1 root root 3378 6月  28 15:19 kube-apiserver.yaml
-rw-------. 1 root root 2879 6月  28 15:19 kube-controller-manager.yaml
-rw-------. 1 root root 1464 6月  28 15:19 kube-scheduler.yaml

# 自定义dashboard yaml文件目录
/opt/kube-dashboard/conf/

总用量 8
-rw-r--r--. 1 root root 1124 6月  29 08:41 admin-user-dashboard.yaml
-rw-r--r--. 1 root root  285 6月  29 08:25 k8s-dashboard.yaml

1. kubernetes 的安装与部署

一、前置工作与注意事项

二、安装

1. 初始准备

2. docker安装

3. k8s安装

三、网络插件的安装

1. 常用网络插件

2. 插件安装

四、安装dashboard

1.注意事项

2.安装

3.权限配置与登录验证

五、子节点加入

六、常用信息

1. 常用命令

2. 环境信息

附录

参考链接

你可能感兴趣的:(1. kubernetes 的安装与部署)