kubernetes1.17.0集群安装ingress-nginx
1.参考列表
https://www.jianshu.com/p/e30b06906b77
https://www.jianshu.com/p/8e3991cedd56
https://www.cnblogs.com/dingbin/p/9754993.html
https://www.cnblogs.com/klvchen/p/9903480.html
https://gitee.com/hmn/ingress-nginx/tree/nginx-0.20.0/deploy
https://www.cnblogs.com/boonya/p/7907999.html
http://www.mamicode.com/info-detail-2825523.html
https://blog.csdn.net/h952520296/article/details/78914036
https://www.xiaoz.me/archives/10578
2.前提
安装了kubernetes集群，参考上篇文章
3.用到的机器（vm模拟）
| 编号 | IP | 配置 | 说明 |
|---|---|---|---|
| 1 | 192.168.117.132 | 2c2g | k8s master节点 |
| 2 | 192.168.117.133 | 8c16g | k8s node节点 |
| 3 | 192.168.117.134 | 1c1g | slb节点 |
| 4 | 192.168.117.1 | 空 | win7机器（vmware宿主机） |
4.整体架构图
其中，LB节点监听80端口，负责TCP 4层转发，代理k8s svc（ingress-nginx）的30080端口。
nginx-ingress-controller即ingress controller，内部也就是一个nginx，k8s中的应用通过创建Ingress类型的对象，修改nginx的配置文件，从而根据Host路由到不同的service。
5.安装ingress controller
kubectl apply -f mandatory.yaml
其中mandatory.yaml的内容是：
apiVersion: v1
kind: Namespace
metadata:
name: ingress-nginx
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: default-http-backend
labels:
app.kubernetes.io/name: default-http-backend
app.kubernetes.io/part-of: ingress-nginx
namespace: ingress-nginx
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: default-http-backend
app.kubernetes.io/part-of: ingress-nginx
template:
metadata:
labels:
app.kubernetes.io/name: default-http-backend
app.kubernetes.io/part-of: ingress-nginx
spec:
terminationGracePeriodSeconds: 60
containers:
- name: default-http-backend
# Any image is permissible as long as:
# 1. It serves a 404 page at /
# 2. It serves 200 on a /healthz endpoint
image: registry.cn-qingdao.aliyuncs.com/kubernetes_xingej/defaultbackend-amd64:1.5
livenessProbe:
httpGet:
path: /healthz
port: 8080
scheme: HTTP
initialDelaySeconds: 30
timeoutSeconds: 5
ports:
- containerPort: 8080
resources:
limits:
cpu: 10m
memory: 20Mi
requests:
cpu: 10m
memory: 20Mi
---
apiVersion: v1
kind: Service
metadata:
name: default-http-backend
namespace: ingress-nginx
labels:
app.kubernetes.io/name: default-http-backend
app.kubernetes.io/part-of: ingress-nginx
spec:
ports:
- port: 80
targetPort: 8080
selector:
app.kubernetes.io/name: default-http-backend
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-configuration
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: tcp-services
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: udp-services
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: nginx-ingress-clusterrole
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- "extensions"
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- "extensions"
resources:
- ingresses/status
verbs:
- update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
name: nginx-ingress-role
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
resourceNames:
# Defaults to "-"
# Here: "-"
# This has to be adapted if you change either parameter
# when launching the nginx-ingress-controller.
- "ingress-controller-leader-nginx"
verbs:
- get
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- ""
resources:
- endpoints
verbs:
- get
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: nginx-ingress-role-nisa-binding
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: nginx-ingress-role
subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: nginx-ingress-clusterrole-nisa-binding
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: nginx-ingress-clusterrole
subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-ingress-controller
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
template:
metadata:
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
annotations:
prometheus.io/port: "10254"
prometheus.io/scrape: "true"
spec:
serviceAccountName: nginx-ingress-serviceaccount
containers:
- name: nginx-ingress-controller
image: registry.cn-qingdao.aliyuncs.com/kubernetes_xingej/nginx-ingress-controller:0.20.0
args:
- /nginx-ingress-controller
- --default-backend-service=$(POD_NAMESPACE)/default-http-backend
- --configmap=$(POD_NAMESPACE)/nginx-configuration
- --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
- --udp-services-configmap=$(POD_NAMESPACE)/udp-services
- --publish-service=$(POD_NAMESPACE)/ingress-nginx
- --annotations-prefix=nginx.ingress.kubernetes.io
securityContext:
capabilities:
drop:
- ALL
add:
- NET_BIND_SERVICE
# www-data -> 33
runAsUser: 33
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
ports:
- name: http
containerPort: 80
- name: https
containerPort: 443
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
---
6.安装一个k8s svc：ingress-nginx，用于暴露pod：nginx-ingress-controller
kubectl apply -f service-nodeport.yaml
service-nodeport.yaml内容如下：
apiVersion: v1
kind: Service
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
type: NodePort
ports:
- name: http
port: 80
targetPort: 80
protocol: TCP
nodePort: 30080
- name: https
port: 443
targetPort: 443
protocol: TCP
nodePort: 30443
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
7.安装一个k8s示例应用kubia
kubectl apply -f kubia.yaml
kubia.yaml内容为：
apiVersion: apps/v1
kind: Deployment
metadata:
name: kubia-web-demo
spec:
selector:
matchLabels:
app: kubia-web-demo
replicas: 1
template:
metadata:
labels:
app: kubia-web-demo
spec:
containers:
- name: kubia
image: registry.cn-hangzhou.aliyuncs.com/jishusc/kubia
ports:
- containerPort: 8380
imagePullSecrets:
- name: registry-secret
---
apiVersion: v1
kind: Service
metadata:
name: kubia-web-demo
spec:
ports:
- port: 8380
targetPort: 8380
selector:
app: kubia-web-demo
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: kubia-web-demo
spec:
rules:
- host: kubia-web.wsc.com
http:
paths:
- path: /
backend:
serviceName: kubia-web-demo
servicePort: 8380
8.测试：通过ingress-nginx svc的nodePort可以访问到kubia应用
8.1 ingress-nginx svc：可以看到通过nodeIP:30080端口可以访问到nginx所在的pod
[root@192 ~]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default-http-backend ClusterIP 10.96.206.99 <none> 80/TCP 13h
ingress-nginx NodePort 10.96.155.148 <none> 80:30080/TCP,443:30443/TCP 89m
8.2 在k8s集群的任一node节点（即在k8s集群内部）上运行curl：可以成功访问到kubia这个应用
[root@192 ~]# curl -H "Host: kubia-web.wsc.com" 192.168.117.132:30080
You have hit kubia-web-demo-6dc9fb9995-j6jrw
8.3 在win7上（即在k8s集群外部）访问kubia应用
配置hosts：其中192.168.117.132机器为k8s master节点所在的机器
192.168.117.132 kubia-web.wsc.com
浏览器访问
9.在机器192.168.117.134上安装单独的nginx，监听80端口，并转发192.168.117.132:30080的流量（TCP四层转发）
yum install gcc-c++
yum install -y pcre pcre-devel
yum install -y zlib zlib-devel
yum install -y openssl openssl-devel
wget http://nginx.org/download/nginx-1.17.6.tar.gz
./configure --with-stream
make
make install
配置/usr/local/nginx/conf/nginx.conf如下：
#user nobody;
worker_processes 1;
#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
#pid logs/nginx.pid;
events {
worker_connections 1024;
}
stream {
server {
listen 80;
proxy_pass 192.168.117.132:30080;
}
}
http {
include mime.types;
default_type application/octet-stream;
#log_format main '$remote_addr - $remote_user [$time_local] "$request" '
# '$status $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';
#access_log logs/access.log main;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
#gzip on;
# another virtual host using mix of IP-, name-, and port-based configuration
#
#server {
# listen 8000;
# listen somename:8080;
# server_name somename alias another.alias;
# location / {
# root html;
# index index.html index.htm;
# }
#}
# HTTPS server
#
#server {
# listen 443 ssl;
# server_name localhost;
# ssl_certificate cert.pem;
# ssl_certificate_key cert.key;
# ssl_session_cache shared:SSL:1m;
# ssl_session_timeout 5m;
# ssl_ciphers HIGH:!aNULL:!MD5;
# ssl_prefer_server_ciphers on;
# location / {
# root html;
# index index.html index.htm;
# }
#}
}
启动nginx
/usr/local/nginx/sbin/nginx
10.最终的测试：在win7机器（k8s集群外部）通过域名访问k8s中的kubia应用
10.1 配置hosts：其中192.168.117.134为nginx所在的机器
192.168.117.134 kubia-web.wsc.com
10.2 浏览器访问
