k8s集群环境搭建以及插件安装
前置条件
终端工具MobaXterm很好用。
1、虚拟机三台（ip按自己的网络环境相应配置）(master/node)
| 节点 | ip |
|---|---|
| k8s-master | 192.168.200.150 |
| k8s-node1 | 192.168.200.151 |
| k8s-node2 | 192.168.200.152 |
2、关闭防火墙(master/node)
systemctl stop firewalld
systemctl disable firewalld
查看防火墙状态：systemctl status firewalld
3、关闭selinux(master/node)
setenforce 0 # 临时关闭
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config # 永久关闭
4、关闭swap(master/node)
swapoff -a # 临时关闭；关闭swap主要是为了性能考虑
free # 可以通过这个命令查看swap是否关闭了
sed -ri 's/.*swap.*/#&/' /etc/fstab # 永久关闭
5、添加主机名与IP对应的关系(master/node)
$ vim /etc/hosts
#添加如下内容：
192.168.200.150 k8s-master
192.168.200.151 k8s-node1
192.168.200.152 k8s-node2
#保存退出
6、修改主机名(master/node)
#k8s-master
[root@localhost ~] hostname
localhost.localdomain
[root@localhost ~] hostname k8s-master ##临时生效
[root@localhost ~] hostnamectl set-hostname k8s-master ##重启后永久生效
#k8s-node1
[root@localhost ~] hostname
localhost.localdomain
[root@localhost ~] hostname k8s-node1 ##临时生效
[root@localhost ~] hostnamectl set-hostname k8s-node1 ##重启后永久生效
7、桥接设置(master/node)
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system
以上几步最好照着都执行一下，以免后面报一大堆错
安装docker(master/node)
注意docker与k8s 的版本对照关系。本次按照的k8s版本为1.18.0，对应的docker版本为docker-ce-19.03.9-3.el7
如果已经安装了dokcer就不需要重复安装了
1.安装wget
$ yum -y install wget
2.添加docker yum源
$ wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O/etc/yum.repos.d/docker-ce.repo
3.安装docker
$ yum -y install docker-ce-19.03.9-3.el7 docker-ce-cli-19.03.9-3.el7
4.设置开机启动
$ systemctl enable docker
5.启动docker
$ systemctl start docker
6.编辑docker配置文件
mkdir /etc/docker/
cat > /etc/docker/daemon.json << EOF
{
"registry-mirrors": ["https://gqs7xcfd.mirror.aliyuncs.com","https://hub-mirror.c.163.com"],
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m"
},
"storage-driver": "overlay2"
}
EOF
7.重启docker
$ systemctl restart docker
安装kubernetes
1.为kubernetes添加阿里云YUM软件源(master/node)
cat > /etc/yum.repos.d/kubernetes.repo << EOF
[k8s]
name=k8s
enabled=1
gpgcheck=0
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
EOF
2.安装kubeadm，kubelet和kubectl(master/node)
版本可以选择自己要安装的版本号，我安装的是1.18.0
$ yum install -y kubelet-1.18.0 kubectl-1.18.0 kubeadm-1.18.0
3.设置开机自启动
$ systemctl enable kubelet
master节点初始化
部署Kubernetes (master) ,node节点不需要执行kubeadm init
kubeadm init \
--apiserver-advertise-address=192.168.200.150 \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.18.0 \
--service-cidr=10.1.0.0/16 \
--pod-network-cidr=10.244.0.0/16
因为初始化的时候会做一下检查，如果出现一下错误。
[preflight] Running pre-flight checks error execution phase preflight:
[preflight] Some fatal errors occurred:
[ERROR Swap]: running with swap on is not supported. Please disable swap [preflight] If you know what you are doing, you can make
a check non-fatal with--ignore-preflight-errors=...To see the
stack trace of this error execute with --v=5 or higher
重启一下就好了，刚才修改的swap没有生效。
初始化完成可以看到下面的画面，如果有错误的话需要排查一下。
配置kubectl
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config
查看节点
kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master NotReady master 6m42s v1.18.0
在两台Node节点上执行join命令
在两台node节点上分别执行join命令
kubeadm join 192.168.200.150:6443 --token 6mjjzr.imfudxt8568utrvv \
--discovery-token-ca-cert-hash sha256:c89fd5ec80eac07429aeb450d475eec53d7d88e51c489c817fd5b7d8f3ebe3ce
查看节点
kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master NotReady master 9m54s v1.18.0
k8s-node1 NotReady 42s v1.18.0
k8s-node2 NotReady 46s v1.18.0
安装插件
1.安装 flannel
从官网下载yaml文件
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
然后执行yaml文件
kubectl apply -f kube-flannel.yml
查看所有pod
kubectl get pods -A
查看命名空间pod
kubectl -n <namespace> get pods -o wide
2.部署busybox来测试集群各网络情况
vi busybox.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: busybox
spec:
replicas: 2
selector:
matchLabels:
name: busybox
template:
metadata:
labels:
name: busybox
spec:
containers:
- name: busybox
image: busybox
imagePullPolicy: IfNotPresent
args:
- /bin/sh
- -c
- sleep 1; touch /tmp/healthy; sleep 30000
readinessProbe:
exec:
command:
- cat
- /tmp/healthy
initialDelaySeconds: 1
kubectl apply -f busybox.yaml
查看所有pod
[root@k8s-master ~]# kubectl get pods -A -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
default busybox-7c84546778-69hq7 1/1 Running 0 90s 10.244.2.2 k8s-node1 <none> <none>
default busybox-7c84546778-plzwg 1/1 Running 0 90s 10.244.1.2 k8s-node2 <none> <none>
进入pod，查看pod 和各node是否互通
kubectl exec -it busybox-7c84546778-69hq7 -- /bin/sh
/ # ping -c 10.244.1.2
ping: invalid number '10.244.1.2'
/ # ping -c -2 10.244.1.2
ping: invalid number '-2'
/ # ping -c 2 10.244.1.2
PING 10.244.1.2 (10.244.1.2): 56 data bytes
64 bytes from 10.244.1.2: seq=0 ttl=62 time=0.504 ms
64 bytes from 10.244.1.2: seq=1 ttl=62 time=1.611 ms
--- 10.244.1.2 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.504/1.057/1.611 ms
/ # ping -c 2 192.168.200.151
PING 192.168.200.151 (192.168.200.151): 56 data bytes
64 bytes from 192.168.200.151: seq=0 ttl=64 time=0.070 ms
64 bytes from 192.168.200.151: seq=1 ttl=64 time=0.074 ms
--- 192.168.200.151 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.070/0.072/0.074 ms
/ # ping -c 2 192.168.200.152
PING 192.168.200.152 (192.168.200.152): 56 data bytes
64 bytes from 192.168.200.152: seq=0 ttl=63 time=0.273 ms
64 bytes from 192.168.200.152: seq=1 ttl=63 time=0.203 ms
--- 192.168.200.152 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.203/0.238/0.273 ms
部署dashboard
下载dashboard
wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.3/aio/deploy/recommended.yaml
#修改dashboard配置，特别要注意空格，要按照前面的对齐！
#vim recommended.yaml
#增加nodeport配置
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
spec:
type: NodePort #增加此行
ports:
- port: 443
targetPort: 8443
nodePort: 30000 #增加此行
selector:
k8s-app: kubernetes-dashboard
kubectl apply -f recommended.yaml
[root@k8s-master ~]# kubectl get pod -n kubernetes-dashboard
NAME READY STATUS RESTARTS AGE
dashboard-metrics-scraper-6b4884c9d5-vmfzc 1/1 Running 0 2m9s
kubernetes-dashboard-7f99b75bf4-2d7sg 1/1 Running 0 2m9s
访问地址：https://192.168.200.150:30000/#/login
创建用户
如上图，跳转到了登录页面，那我们就先创建个用户：
1.创建服务账号
首先创建一个叫admin-user的服务账号，并放在kube-system名称空间下：
admin-user.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: admin-user
namespace: kube-system
执行kubectl create命令：
kubectl create -f admin-user.yaml
2.绑定角色
默认情况下，kubeadm创建集群时已经创建了admin角色，我们直接绑定即可：
admin-user-role-binding.yaml
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: admin-user
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: admin-user
namespace: kube-system
执行kubectl create命令：
kubectl create -f admin-user-role-binding.yaml
3.获取Token
现在我们需要找到新创建的用户的Token，以便用来登录dashboard：
kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')
输出类似：
Name: admin-user-token-qrj82
Namespace: kube-system
Labels: <none>
Annotations: kubernetes.io/service-account.name=admin-user
kubernetes.io/service-account.uid=6cd60673-4d13-11e8-a548-00155d000529
Type: kubernetes.io/service-account-token
Data
====
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi11c2VyLXRva2VuLXFyajgyIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiI2Y2Q2MDY3My00ZDEzLTExZTgtYTU0OC0wMDE1NWQwMDA1MjkiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06YWRtaW4tdXNlciJ9.C5mjsa2uqJwjscWQ9x4mEsWALUTJu3OSfLYecqpS1niYXxp328mgx0t-QY8A7GQvAr5fWoIhhC_NOHkSkn2ubn0U22VGh2msU6zAbz9sZZ7BMXG4DLMq3AaXTXY8LzS3PQyEOCaLieyEDe-tuTZz4pbqoZQJ6V6zaKJtE9u6-zMBC2_iFujBwhBViaAP9KBbE5WfREEc0SQR9siN8W8gLSc8ZL4snndv527Pe9SxojpDGw6qP_8R-i51bP2nZGlpPadEPXj-lQqz4g5pgGziQqnsInSMpctJmHbfAh7s9lIMoBFW7GVE8AQNSoLHuuevbLArJ7sHriQtDB76_j4fmA
ca.crt: 1025 bytes
namespace: 11 bytes
然后把Token复制到登录界面的Token输入框中，登入后显示如下：
部署metrics-server
下载metrics-server（会抽风下载不下来）
wget https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
修改镜像地址
将部署文件中镜像地址修改为国内的地址，大概在部署文件的第140行。 原配置是：
image: k8s.gcr.io/metrics-server/metrics-server:v0.6.2
修改后的配置是：
image: registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-server:v0.6.2
部署metrics server
kubectl create -f components.yaml
查看metric server的运行情况，发现探针问题：Readiness probe failed: HTTP probe failed with statuscode: 500
[root@centos05 deployment]# kubectl get pods -n kube-system | grep metrics
kube-system metrics-server-7f6b85b597-j2p5h 0/1 Running 0 2m23s
[root@centos05 deployment]# kubectl describe pod metrics-server-7f6b85b597-j2p5h -n kube-system
在大概 139 行的位置追加参数：–kubelet-insecure-tls，修改后内容如下：
spec:
containers:
- args:
- --cert-dir=/tmp
- --secure-port=4443
- --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
- --kubelet-use-node-status-port
- --metric-resolution=15s
- --kubelet-insecure-tls
再次部署文件：
kubectl apply -f components.yaml
执行kubectl top命令成功：
[root@k8s-master kubernetes]# kubectl top pod
NAME CPU(cores) MEMORY(bytes)
busybox-7c84546778-69hq7 1m 0Mi
busybox-7c84546778-plzwg 1m 0Mi
nginx-deployment 0m 1Mi
nginx-deployment-788b8d7b98-528cj 0m 1Mi
nginx-deployment-788b8d7b98-8j6q4 0m 1Mi
再进入dashboard发现有监控了，token过期的话再执行上面获取token一次。
