上一回留下一个问题，就是使用浏览器无法访问到，这一回继续排错。
1. 排错思路整理
1) nginx-ingress-controller
pod nginx-ingress-controller-6d96ccd6f4-g2d6b , master与node 都可以访问
kubectl describe pod nginx-ingress-controller-6d96ccd6f4-g2d6b -n ingress-nginx
Name: nginx-ingress-controller-6d96ccd6f4-g2d6b
Namespace: ingress-nginx
Node: 192.168.89.133/192.168.89.133
Start Time: Wed, 08 Jan 2020 09:34:34 +0800
Labels: app.kubernetes.io/name=ingress-nginx
app.kubernetes.io/part-of=ingress-nginx
pod-template-hash=6d96ccd6f4
Annotations: kubernetes.io/limit-ranger: LimitRanger plugin set: cpu, memory request for container nginx-ingress-controller
prometheus.io/port: 10254
prometheus.io/scrape: true
Status: Running
IP: 10.1.20.3 # 这个为nginx-ingress-controller pod 的ip
IPs:
IP: 10.1.20.3
Controlled By: ReplicaSet/nginx-ingress-controller-6d96ccd6f4
Containers:
nginx-ingress-controller:
Container ID: docker://20d0722863e8896d830b43f730c8b233009e0a81c9ab1841f805a761ca5d235e
Image: 192.168.89.132:5000/nginx-ingress-controller:0.26.2
Image ID: docker-pullable://192.168.89.132:5000/nginx-ingress-controller@sha256:417cf5edc00c1581a44f67c1b1713b55db3811834498f45fc5de40fbe5f6a8de
Ports: 80/TCP, 443/TCP
Host Ports: 0/TCP, 0/TCP
Args:
/nginx-ingress-controller
--configmap=$(POD_NAMESPACE)/nginx-configuration
--tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
--udp-services-configmap=$(POD_NAMESPACE)/udp-services
--publish-service=$(POD_NAMESPACE)/ingress-nginx
--annotations-prefix=nginx.ingress.kubernetes.io
State: Running
Started: Wed, 08 Jan 2020 14:27:53 +0800
Last State: Terminated
Reason: Error
Exit Code: 137
Started: Wed, 08 Jan 2020 09:35:24 +0800
Finished: Wed, 08 Jan 2020 14:27:40 +0800
Ready: True
Restart Count: 1
Requests:
cpu: 100m
memory: 90Mi
Liveness: http-get http://:10254/healthz delay=10s timeout=10s period=10s #success=1 #failure=3 , 健康性检测 curl http://10.1.20.3:10254/healthz
Readiness: http-get http://:10254/healthz delay=0s timeout=10s period=10s #success=1 #failure=3
Environment:
POD_NAME: nginx-ingress-controller-6d96ccd6f4-g2d6b (v1:metadata.name)
POD_NAMESPACE: ingress-nginx (v1:metadata.namespace)
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from nginx-ingress-serviceaccount-token-mz87t (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
nginx-ingress-serviceaccount-token-mz87t:
Type: Secret (a volume populated by a Secret)
SecretName: nginx-ingress-serviceaccount-token-mz87t
Optional: false
QoS Class: Burstable
Node-Selectors: kubernetes.io/os=linux
Tolerations:
Events:
curl http://10.1.20.3:10254/healthz
ok
过了一会儿只有89.133这一台可以访问，奇怪! 说明控制器在正常工作，没有问题，不过只有一台可以正常访问接口，不知道这是不是正常的。
kubectl get svc -n ingress-nginx -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
ingress-nginx NodePort 169.169.178.15 80:30080/TCP,443:30443/TCP 164m app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx
service类型为NodePort , service ip : 169.169.178.15 , 该ip两台node可以ping通，master无法ping通 ， 该地址为nginx-ingress-controller pod对应的service
kubectl describe svc ingress-nginx -n ingress-nginx
Name: ingress-nginx
Namespace: ingress-nginx
Labels: app.kubernetes.io/name=ingress-nginx
app.kubernetes.io/part-of=ingress-nginx
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app.kubernetes.io/name":"ingress-nginx","app.kubernetes.io/par...
Selector: app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx
Type: NodePort
IP: 169.169.178.15
Port: http 80/TCP
TargetPort: 80/TCP
NodePort: http 30080/TCP
Endpoints: 10.1.20.3:80
Port: https 443/TCP
TargetPort: 443/TCP
NodePort: https 30443/TCP
Endpoints: 10.1.20.3:443
Session Affinity: None
External Traffic Policy: Cluster
Events:
curl http://169.169.178.15
404 Not Found
404 Not Found
openresty/1.15.8.2
curl http://169.169.178.15/healthz 回应为空 , 除了 /healthz ， 访问其他都报404
2). ingress.yaml 规则
less ingress.yaml
spec:
# 路由规则
rules:
# 主机名，只能是域名，修改为你自己的
- host: k8s.aaa.com
http:
paths:
- path:
backend:
# 后台部署的 Service Name，与上面部署的 Tomcat 对应
serviceName: tomcat-http
# 后台部署的 Service Port，与上面部署的 Tomcat 对应
servicePort: 8080
配置文件最下面部分可知，访问 k8s.aaa.com ，跳转到service名为tomcat-http，端口为8080的服务，思考：这个k8s.aaa.com映射和谁一起？
kubectl get Ingress
NAME HOSTS ADDRESS PORTS AGE
nginx-web k8s.aaa.com 169.169.178.15 80 21h
ADDRESS 169.169.178.15为nginx-ingress的服务地址 ， 由此可知，Ingress.yaml没有创建容器，而只是创建了一种路由规则，给这个路由规则起了一个名字: nginx-web
k8s.aaa.com -> 169.169.178.15 -> 10.1.20.3 -> 起了一个nginx容器，upstream定义了调度规则 -> tomcat-http -> 169.169.70.125(tomcat-http服务ip) -> tomcat-app[ 10.1.27.4(192.168.89.134)/10.1.20.5(192.168.89.133) ] -> 访问到tomcat服务
为了不引起干扰，把master和两台node /etc/hosts里有关k8s.aaa.com的映射都删了。
k8snode01: ip a
6: kube-ipvs0: mtu 1500 qdisc noop state DOWN group default
link/ether fe:fb:9b:d5:31:d0 brd ff:ff:ff:ff:ff:ff
inet 169.169.0.1/32 brd 169.169.0.1 scope global kube-ipvs0
valid_lft forever preferred_lft forever
inet 169.169.0.10/32 brd 169.169.0.10 scope global kube-ipvs0
valid_lft forever preferred_lft forever
inet 169.169.70.125/32 brd 169.169.70.125 scope global kube-ipvs0
valid_lft forever preferred_lft forever
inet 169.169.178.15/32 brd 169.169.178.15 scope global kube-ipvs0
valid_lft forever preferred_lft forever
发现这个ip 169.169.178.15 在192.168.89.133这台机器上被解析在kube-ipvs0上了。
思考： 访问k8s.aaa.com ， 就是访问169.169.178.15 , 就是访问192.168.89.133 ?
找一台机器：dnsserver(192.168.89.128):
/etc/hosts:
192.168.89.133 k8s.aaa.com
curl k8s.aaa.com
curl: (7) Failed connect to k8s.aaa.com:80; Connection refused
不行，这样解析不行，咋弄？
k8snode01:
curl -D- http://169.169.178.15 -H 'Host: k8s.aaa.com'
可以出来
将192.168.89.133 k8s.aaa.com 写到windows hosts 里；
访问加上30080端口，
http://k8s.aaa.com:30080/
终于出来了，访问的时候是要加端口的
3). 为了验证一下是不是这个问题，在测试一个服务
vim test-ingress-pod.yaml
apiVersion: v1
kind: Service
metadata:
name: myapp-svc
namespace: default
spec:
selector:
app: myapp
env: test
ports:
- name: http
port: 80
targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp-test
spec:
replicas: 2
selector:
matchLabels:
app: myapp
env: test
template:
metadata:
labels:
app: myapp
env: test
spec:
containers:
- name: myapp
image: nginx:1.15-alpine
ports:
- name: httpd
containerPort: 80
vim test-ingress-myapp.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-myapp
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: www.tchua.top
http:
paths:
- path: /
backend:
serviceName: myapp-svc
servicePort: 80
kubectl create -f test-ingress-pods.yaml
kubectl create -f test-ingress-myapp.yaml
windows 里配置 192.168.89.133 www.tchua.top
浏览器访问： http://www.tchua.top:30080 , nginx页面出来了，问题解决
2. 总结
有些问题出现了，一路排错排不出来，可能参考一下其他人的文档，灵感思路就来了。此处要点：这里访问时需要加上svc映射到主机时产生的nodePort端口号，service-nodeport.yaml中定义
- name: http
port: 80
targetPort: 80
protocol: TCP
nodePort: 30080
- name: https
port: 443
targetPort: 443
protocol: TCP
nodePort: 30443
有时候就是一个小小的问题，就能卡好几天，一定不要急躁，休息一下，忙点其他东西，再一看，就出来了，另外，对应用的原理还是要特别熟悉，再整理下这个ingress流程：
ingress是为了给集群外部暴露服务使用的，首先要部署好ingress controller 控制器，我们选择的是nginx ingress controller，官网上有对应的搭建流程，nginx-ingress-controller镜像比较大，直接构建比较耗时，可以提前下载好导入本地仓库。控制器弄好后，创建测试应用，pod,services等部署好后，再部署ingress.yaml，这个文件编写的是调度规则，给nginx controller控制器使用，要查看ingress controller控制器在哪一台，以及这个服务映射到节点上的端口，进行解析，浏览器访问的时候需要加上端口号。
因为上面为测试用例，如果在真实环境，可以将新的域名解析到公司内部dns服务器上，访问的时候还要使用端口号，这个问题也好解决，在其前部加一个nginx调度，使用标准端口调度到后面的指定的解析端口上即可。
本地应用案例流程：
a. 将tchua.top域名解析到公司内部dns服务器上
dnsserver:
vim /etc/named.conf
......
zone "tchua.top" IN {
type master;
file "tchua.top.zone";
};
......
cp -a /var/named/web.com.zone /var/named/tchua.top.zone
vim /var/named/tchua.top.zone
$TTL 1D
@ IN SOA @ tchua.top. (
0 ; serial
1D ; refresh
1H ; retry
1W ; expire
3H ) ; minimum
NS @
A 192.168.89.133
www A 192.168.89.133
systemctl restart named
hosts文件即可去掉，直接浏览器访问 http://www.tchua.top:30080
b. 去掉端口30080访问
修改映射端口http://:30080到常用端口80
vim service-nodeport.yaml
...
- name: http
port: 80
targetPort: 80
protocol: TCP
nodePort: 80
...
kubectl apply -f service-nodeport.yaml
kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx NodePort 169.169.178.15 80:80/TCP,443:30443/TCP 5h7m
发现端口已修改
浏览器访问： http://k8s.aaa.com 和 http://www.tchua.top 都可以访问到，验证通过，可以正常使用
