一、系统环境
CentOS Linux release 7.9.2009 (Core)
kubectl-1.20.4-0.x86_64
kubelet-1.20.4-0.x86_64
kubeadm-1.20.4-0.x86_64
kubernetes-cni-0.8.7-0.x86_64
二、k8s架构
| 用途 | ip地址 | 主机名 |
| master | 192.168.10.127 | minio-4 |
| node01 | 192.168.10.124 | minio-1 |
| node02 | 192.168.10.125 | minio-2 |
| node03 | 192.168.10.126 | minio-3 |
| nfs存储 | 192.168.10.143 | |
三、Ingress Controller原理
具体可以参考：深入理解ingress-nginx - 赵SIR - 博客园
Ingress Controller有很多实现，我们这里采用官方维护的Nginx控制器。
部署文档：https://github.com/kubernetes/ingress-nginx/blob/master/docs/deploy/index.md
四、具体安装方法
1、下载官方nignx版本
wget -O nginx-ingress.yaml https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.41.2/deploy/static/provider/cloud/deploy.yaml
image.gif
2、修改下载后的nginx-ingress.yaml
vim nginx-ingress.yaml
image.gif
3、增加行 replicas: 2 副本数
spec:
selector:
matchLabels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/component: controller
revisionHistoryLimit: 10
strategy:
rollingUpdate:
maxUnavailable: 1
type: RollingUpdate
minReadySeconds: 0
replicas: 2
template:
metadata:
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/component: controller
spec:
hostNetwork: true
dnsPolicy: ClusterFirst
image.gif
4、修改镜像为国内镜像
增加行 hostNetwork: true
image: pollyduan/ingress-nginx-controller:v0.41.2
minReadySeconds: 0
replicas: 2
template:
metadata:
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/component: controller
spec:
hostNetwork: true
dnsPolicy: ClusterFirst
containers:
- name: controller
image: pollyduan/ingress-nginx-controller:v0.41.2
#image: k8s.gcr.io/ingress-nginx/controller:v0.48.1@sha256:e9fb216ace49dfa4a5983b183067e97496e7a8b307d2093f4278cd550c303899
imagePullPolicy: IfNotPresent
lifecycle:
image.gif
5、节点服务器上添加标签 和 ingress-deploy.yaml 中保持一致否则部署时会出现错误
image
image.gif
在每个节点执行打标签命令
#kubectl label nodes master ingress-ready=true
#kubectl label nodes master kubernetes.io/os=linux --overwrite
image.gif
查看label,应该就有刚才执行的两个标签
# kubectl get nodes --show-labels
NAME STATUS ROLES AGE VERSION LABELS
minio-1 Ready master 118d v1.20.4 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,ingress-ready=true,kubernetes.io/arch=amd64,kubernetes.io/hostname=minio-1,kubernetes.io/os=linux,node-role.kubernetes.io/master=true
minio-2 Ready master 118d v1.20.4 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,ingress-ready=true,kubernetes.io/arch=amd64,kubernetes.io/hostname=minio-2,kubernetes.io/os=linux,node-role.kubernetes.io/master=true
minio-3 Ready master 101d v1.20.4 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,ingress-ready=true,kubernetes.io/arch=amd64,kubernetes.io/hostname=minio-3,kubernetes.io/os=linux,node-role.kubernetes.io/master=true
minio-4 Ready control-plane,master 118d v1.20.4 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,ingress-ready=false,kubernetes.io/arch=amd64,kubernetes.io/hostname=minio-4,kubernetes.io/os=linux,node-role.kubernetes.io/control-plane=,node-role.kubernetes.io/master=
image.gif
image
image.gif
6、部署ingress-nginx服务
#kubectl create -f nginx-ingress.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
configmap/ingress-nginx-controller created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
service/ingress-nginx-controller-admission created
service/ingress-nginx-controller created
deployment.apps/ingress-nginx-controller created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
serviceaccount/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
image.gif
查看运行情况
# kubectl get pod,service -n ingress-nginx -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/ingress-nginx-admission-create-bwg5x 0/1 Completed 0 72d 10.244.2.39 minio-2
pod/ingress-nginx-admission-patch-958vt 0/1 Completed 0 72d 10.244.1.31 minio-1
pod/ingress-nginx-controller-6ff975b7f7-2dcjh 1/1 Running 1 72d 192.168.10.126 minio-3
pod/ingress-nginx-controller-6ff975b7f7-fwlql 1/1 Running 1 72d 192.168.10.124 minio-1
pod/ingress-nginx-controller-6ff975b7f7-wdhkw 1/1 Running 1 72d 192.168.10.125 minio-2
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/ingress-nginx-controller NodePort 10.10.26.171 80:30096/TCP,443:30113/TCP 72d app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
service/ingress-nginx-controller-admission ClusterIP 10.10.3.13 443/TCP 72d app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
image.gif
#kubectl get pods -n ingress-nginx
NAME READY STATUS RESTARTS AGE
ingress-nginx-admission-create-bwg5x 0/1 Completed 0 72d
ingress-nginx-admission-patch-958vt 0/1 Completed 0 72d
ingress-nginx-controller-6ff975b7f7-2dcjh 1/1 Running 1 72d
ingress-nginx-controller-6ff975b7f7-fwlql 1/1 Running 1 72d
ingress-nginx-controller-6ff975b7f7-wdhkw 1/1 Running 1 72d
image.gif
如果pod状态有异样请使用kubectl describe pod pod名称 -n ingress-nginx 查看详情
#kubectl describe pod ingress-nginx-controller-6ff975b7f7-2dcjh -n ingress-nginx
Name: ingress-nginx-controller-6ff975b7f7-2dcjh
Namespace: ingress-nginx
Priority: 0
Node: minio-3/192.168.10.126
Start Time: Wed, 11 Aug 2021 13:52:07 +0800
Labels: app.kubernetes.io/component=controller
app.kubernetes.io/instance=ingress-nginx
app.kubernetes.io/name=ingress-nginx
pod-template-hash=6ff975b7f7
Annotations:
Status: Running
IP: 192.168.10.126
IPs:
IP: 192.168.10.126
Controlled By: ReplicaSet/ingress-nginx-controller-6ff975b7f7
Containers:
controller:
Container ID: docker://9fcedbe86046d9dda1f3f26778efcd38ede942e341fc65dec1a83a07c5822e26
Image: pollyduan/ingress-nginx-controller:v0.41.2
Image ID: docker-pullable://pollyduan/ingress-nginx-controller@sha256:c7f8c2eaa157cc18f267c6c89a9e659f65ee7b347547a37c6471aff78ca22021
Ports: 80/TCP, 443/TCP, 8443/TCP
Host Ports: 80/TCP, 443/TCP, 8443/TCP
Args:
/nginx-ingress-controller
--election-id=ingress-controller-leader
--ingress-class=nginx
--configmap=$(POD_NAMESPACE)/ingress-nginx-controller
--validating-webhook=:8443
--validating-webhook-certificate=/usr/local/certificates/cert
--validating-webhook-key=/usr/local/certificates/key
--publish-status-address=localhost
State: Running
Started: Wed, 15 Sep 2021 09:20:42 +0800
Ready: True
Restart Count: 1
Requests:
cpu: 100m
memory: 90Mi
Liveness: http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=5
Readiness: http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=3
Environment:
POD_NAME: ingress-nginx-controller-6ff975b7f7-2dcjh (v1:metadata.name)
POD_NAMESPACE: ingress-nginx (v1:metadata.namespace)
LD_PRELOAD: /usr/local/lib/libmimalloc.so
Mounts:
/usr/local/certificates/ from webhook-cert (ro)
/var/run/secrets/kubernetes.io/serviceaccount from ingress-nginx-token-sjjs6 (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
webhook-cert:
Type: Secret (a volume populated by a Secret)
SecretName: ingress-nginx-admission
Optional: false
ingress-nginx-token-sjjs6:
Type: Secret (a volume populated by a Secret)
SecretName: ingress-nginx-token-sjjs6
Optional: false
QoS Class: Burstable
Node-Selectors: ingress-ready=true
kubernetes.io/os=linux
Tolerations: node-role.kubernetes.io/master:NoSchedule
node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
image.gif
6、测试一个服务
#前提你的nginx服务已经部署好
vim pvc.test.cn-ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: pvc.test.cn
# 命名空间和代理的serviceName 所属命名空间保存一致，否则访问是会出现503错误
namespace: default
spec:
rules:
- host: pvc.test.cn
http:
paths:
- backend:
# 注册的服务名称
serviceName: nginx-pvc-php74
# 服务端口
servicePort: 80
image.gif
#kubectl create -f pvc.test.cn-ingress.yaml
image.gif
查看状态
kubectl get ingress --all-namespaces
NAMESPACE NAME CLASS HOSTS ADDRESS PORTS AGE
default pvc.test.cn pvc.test.cn localhost 80 44d
image.gif
在dns做一下域名解析
因为我这3台的node都装了ingress服务，所以dns上就解析了3条记录，这样就可以轮询
pvc A 192.168.10.124
pvc A 192.168.10.125
pvc A 192.168.10.126
image.gif