Ingress是一个API对象，用来管理集群外部访问集群内部的服务（主要为http和https）。Ingress可以提供负载均衡、ssl卸载和虚拟主机的功能。可以理解service是4层负载均衡，ingress是7层负载均衡。
Ingress控制器
Ingress控制器不同于其他控制，他不属于控制器管理器，而是一个类似coreDns的插件，常用的ingress控制器类型有：
Contour(envoy)、F5、Haproxy、ISTIO、Kong、Nginx、Traefik
可以在集群中部署任意数量的ingress控制器，其实就是部署多个Deployment或者DaemonSet。创建ingress资源时使用ingress-class注释annotate使用哪个ingress controller。
Ingress资源
与其他资源相同，ingress资源也有apiVersion、kind和metadata三个字段。不同的ingress控制器支持不同的annotations，ingress经常使用annotations来配置一些选项，例如rewrite-target。了解支持哪些注释需要查看相关文档。Ingress spec包含配置负载均衡器或代理服务器所需的所有信息。ingress资源仅支持用于HTTP的规则，每个HTTP规则都包含以下信息：
主机（可选）：适用于虚拟主机
路径列表（例如，/testpath），每个路径都有一个用servicename和serviceport定义的相关后端。在LoadBalancer将流量导向引用的服务之前，主机和路径必须与传入请求的内容匹配。
后端是服务和端口名称的组合。对与主机和规则路径匹配的入口的HTTP（和HTTPS）请求将发送到列出的后端。
缺省后端ingress.spec.backend，未匹配到任何rule的流量，将路由至缺省后端
下载并修改ingress相关资源的yaml文件
wget https://raw.githubusercontent.com/nginxinc/kubernetes-ingress/master/deployments/common/ns-and-sa.yaml
wget https://raw.githubusercontent.com/nginxinc/kubernetes-ingress/master/deployments/common/nginx-config.yaml
wget https://raw.githubusercontent.com/nginxinc/kubernetes-ingress/master/deployments/rbac/rbac.yaml
wget https://raw.githubusercontent.com/nginxinc/kubernetes-ingress/master/deployments/daemon-set/nginx-ingress.yaml
default-server-secret.yaml是缺省证书的配置，如果自己导入证书则不需要下载，在这里自己导入证书
wget https://raw.githubusercontent.com/nginxinc/kubernetes-ingress/master/deployments/common/default-server-secret.yaml
nodeport.yaml是访问ingress service的方式，如果需要可以下载，这里采用的是hostNetwork的方式
wget https://raw.githubusercontent.com/nginxinc/kubernetes-ingress/master/deployments/service/nodeport.yaml
如果采用nodeport方式访问，建议修改nodeport.yaml Service.spec.ports.[*].nodePort，固定监听端口
apiVersion: v1
kind: Service
metadata:
name: nginx-ingress
namespace: nginx-ingress
spec:
type: NodePort
ports:
- port: 80
targetPort: 80
protocol: TCP
name: http
#使用固定nodePort
nodePort: 30080
- port: 443
targetPort: 443
protocol: TCP
name: https
#使用固定nodePort
nodePort: 30443
selector:
app: nginx-ingress
修改DaemonSet nginx-ingress的配置，使用带有标签app=nginx-ingress的Node，并使用Node的网络hostNetwork: True
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: nginx-ingress
namespace: nginx-ingress
spec:
selector:
matchLabels:
app: nginx-ingress
template:
metadata:
labels:
app: nginx-ingress
#annotations:
#prometheus.io/scrape: "true"
#prometheus.io/port: "9113"
spec:
#使用特定的node
nodeSelector:
app: ingress
#使用node的网络
hostNetwork: true
serviceAccountName: nginx-ingress
containers:
- image: nginx/nginx-ingress:edge
imagePullPolicy: Always
name: nginx-ingress
ports:
- name: http
containerPort: 80
hostPort: 80
- name: https
containerPort: 443
hostPort: 443
#- name: prometheus
#containerPort: 9113
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
args:
- -nginx-configmaps=$(POD_NAMESPACE)/nginx-config
- -default-server-tls-secret=$(POD_NAMESPACE)/default-server-secret
#- -v=3 # Enables extensive logging. Useful for troubleshooting.
#- -report-ingress-status
#- -external-service=nginx-ingress
#- -enable-leader-election
#- -enable-prometheus-metrics
#- -enable-custom-resources
生成相关证书
生成ca证书
openssl genrsa -out ca.key 4096
openssl req -new -key ca.key -out ca.csr -set_serial 0 -subj "/CN=cdrcb-ingress-ca"
openssl x509 -req -days 7300 -in ca.csr -signkey ca.key -out ca.crt -extfile /etc/pki/tls/openssl.cnf -extensions v3_req -extensions v3_ca
生成default.crt
openssl genrsa -out default.key 4096
openssl req -new -key default.key -out default.csr -subj "/CN=cdrcb-ingress-default"
openssl x509 -req -days 3650 -sha1 -extensions v3_req -CA ca.crt -CAkey ca.key -CAserial ca.srl -CAcreateserial -in default.csr -out default.crt
生成test1.crt
openssl genrsa -out test1.key 4096
openssl req -new -key test1.key -out test1.csr -subj "/CN=cdrcb-ingress-test1"
openssl x509 -req -days 3650 -sha1 -extensions v3_req -CA ca.crt -CAkey ca.key -CAserial ca.srl -CAcreateserial -in test1.csr -out test1.crt
生成test2.crt
openssl genrsa -out test2.key 4096
openssl req -new -key test2.key -out test2.csr -subj "/CN=cdrcb-ingress-test2"
openssl x509 -req -days 3650 -sha1 -extensions v3_req -CA ca.crt -CAkey ca.key -CAserial ca.srl -CAcreateserial -in test2.csr -out test2.crt
创建ingress controller相关资源
#创建namespace
kubectl apply -f ns-and-sa.yaml
#导入证书，生成tls类型的secret
kubectl create secret tls default-server-secret --cert=default.crt --key=default.key
kubectl create secret tls cdrcb-ingress-test1 --cert=test1.crt --key=test1.key
kubectl create secret tls cdrcb-ingress-test2 --cert=test2.crt --key=test2.key
#创建ingress controller
kubectl apply -f nginx-config.yaml
kubectl apply -f rbac.yaml
kubectl apply -f nginx-ingress.yaml
kubectl apply -f nodeport.yaml
创建后端service test1\test2及相关pod
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: busybox-httpd-1
spec:
replicas: 2
selector:
matchLabels:
run: busybox-httpd-1
template:
metadata:
labels:
run: busybox-httpd-1
spec:
containers:
- command:
- /bin/sh
- -c
- ifconfig > index.html && /bin/httpd -f
image: busybox
name: busybox-httpd
---
apiVersion: v1
kind: Service
metadata:
name: test1
spec:
selector:
run: busybox-httpd-1
ports:
- name: http
port: 80
protocol: TCP
targetPort: 80
type: ClusterIP
---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: busybox-httpd-2
spec:
replicas: 2
selector:
matchLabels:
run: busybox-httpd-2
template:
metadata:
labels:
run: busybox-httpd-2
spec:
containers:
- command:
- /bin/sh
- -c
- ifconfig > index.html && /bin/httpd -f
image: busybox
name: busybox-httpd
---
apiVersion: v1
kind: Service
metadata:
name: test2
spec:
selector:
run: busybox-httpd-2
ports:
- name: http
port: 80
protocol: TCP
targetPort: 80
type: ClusterIP
创建ingress资源
client(https://test1) ---- ingress ---- test1:80
client(https://test2) ---- ingress ---- test2:80
annotations是实现ingress功能的重要手段：
例如：存在多种ingress controller时，使用
annotations：
kubernetes.io/ingress.class: nginx
来表示使用哪一种ingress controller.
通常要添加
annotations：
nginx.ingress.kubernetes.io/rewrite-target: /
实现nginx的rewrite-target
使用如下yaml来创建ingress资源
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-test
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- hosts:
- test1
secretName: cdrcb-ingress-test1
- hosts:
- test2
secretName: cdrcb-ingress-test2
rules:
- host: test1
http:
paths:
- path: /
backend:
serviceName: test1
servicePort: 80
- host: test2
http:
paths:
- path: /
backend:
serviceName: test2
servicePort: 80
查看ingress controller可以看到实际使用的pod的ip
kubectl describe ingresses ingress-test
Name: ingress-test
Namespace: default
Address:
Default backend: default-http-backend:80 ()
TLS:
cdrcb-ingress-test1 terminates test1
cdrcb-ingress-test2 terminates test2
Rules:
Host Path Backends
---- ---- --------
test1
/ test1:80 (10.244.196.131:80,10.244.241.202:80)
test2
/ test2:80 (10.244.140.66:80,10.244.186.195:80)
查看nginx配置
kubectl exec -n nginx-ingress -it nginx-ingress-sv6fg /bin/bash
root@ingressnode01:/# cat /etc/nginx/conf.d/default-ingress-test.conf
# configuration for default/ingress-test
upstream default-ingress-test-test1-test1-80 {
random two least_conn;
server 10.244.196.131:80 max_fails=1 fail_timeout=10s;
server 10.244.241.202:80 max_fails=1 fail_timeout=10s;
}
upstream default-ingress-test-test2-test2-80 {
random two least_conn;
server 10.244.140.66:80 max_fails=1 fail_timeout=10s;
server 10.244.186.195:80 max_fails=1 fail_timeout=10s;
}
server {
listen 80;
listen 443 ssl;
ssl_certificate /etc/nginx/secrets/default-cdrcb-ingress-test1;
ssl_certificate_key /etc/nginx/secrets/default-cdrcb-ingress-test1;
server_tokens on;
server_name test1;
if ($scheme = http) {
return 301 https://$host:443$request_uri;
}
location / {
proxy_http_version 1.1;
proxy_connect_timeout 60s;
proxy_read_timeout 60s;
client_max_body_size 1m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering on;
proxy_pass http://default-ingress-test-test1-test1-80;
}
}
server {
listen 80;
listen 443 ssl;
ssl_certificate /etc/nginx/secrets/default-cdrcb-ingress-test2;
ssl_certificate_key /etc/nginx/secrets/default-cdrcb-ingress-test2;
server_tokens on;
server_name test2;
if ($scheme = http) {
return 301 https://$host:443$request_uri;
}
location / {
proxy_http_version 1.1;
proxy_connect_timeout 60s;
proxy_read_timeout 60s;
client_max_body_size 1m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering on;
proxy_pass http://default-ingress-test-test2-test2-80;
}
}
验证效果
image.png
image.png