亚马逊 eks 集群代理用了traefik alb绑定 traefik 在node 上面暴露的端口,对外提供服务。一直也没有在意是否能正常获取客户端IP(腾讯云 阿里云的traefik 后端应用都能正常获取客户端IP),后端php服务基础镜像基于https://github.com/richarvey/nginx-php-fpm构建,对日志进行了json格式化,安装了一些其他依赖。最近后端小伙伴有个需求要获取到客户端IP然后上传给第三方,这才发现xff字段是内网IP,大概就是这样的:
{"@timestamp_local":"2023-06-27T10:37:10+00:00","host":"10.0.36.94","clientip":"10.0.37.83","size":552,"responsetime":0.000,"upstreamtime":"0.000","upstreamhost":"127.0.0.1:1215","http_host":"xxx.xxxx.com","url":"/","type":"newnginx-api","request":"GET / HTTP/1.1","time_local":"27/Jun/2023:10:37:10 +0000","xff":"10.0.16.130","city_name":"-","country_code2":"-","country_name":"-","latitude":"-","longitude":"-","region_name":"-","region_code":"-","referer":"-","agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36","status":"502"}
注:图文不符阿,两个环境!但是演示效果一样!
xff字段获取的是内网IP怎么破?如何获取客户端真实IP呢?
点开属性面板:
我也尝试了附加?貌似不可以,保留会显示两个IP但是可以显示客户端真实IP!
traefik的部署方式基本是Kubernetes 1.20.5 安装traefik在腾讯云下的实践这样的。当然了crd rbac等用了官方新的配置文件!
traefik修改配置参照:https://doc.traefik.io/traefik/routing/entrypoints/#forwarded-headers
traefik-config.yaml
web:
address: ":80" ## 配置 80 端口,并设置入口名称为 web
forwardedHeaders:
insecure: true
proxyProtocol:
insecure: true
注: alb https也直接挂载了后端http 端口(web端口),故只做了web相关配置,这里根据实际需求进行配置!
kubectl apply -f traefik-config.yaml -n traefik
删除Pod 使Pod重新加载configmap文件!当然了如果有更优雅的方式也可以使用,我这里就暴力一下了!
[root@ip-10-0-13-199 yaml]# kubectl get pods -n traefik
NAME READY STATUS RESTARTS AGE
traefik-ingress-controller-229bm 1/1 Running 0 32m
traefik-ingress-controller-n9fzv 1/1 Running 0 32m
traefik-ingress-controller-vlbqs 1/1 Running 0 32m
[root@ip-10-0-13-199 yaml]# kubectl delete pods traefik-ingress-controller-229bm traefik-ingress-controller-n9fzv traefik-ingress-controller-vlbqs -n traefik
pod "traefik-ingress-controller-229bm" deleted
pod "traefik-ingress-controller-n9fzv" deleted
pod "traefik-ingress-controller-vlbqs" deleted
继续浏览器访问后端web项目域名:
恩xff字段可以显示客户端公网IP了!就先这样了
另外还有一个集群的traefik helm配置的该怎么搞呢?个人觉得应该是这样:https://community.traefik.io/t/forwardedheaders-insecure-to-true-not-working/11092/4
修改value.yaml中配置:
- "--entryPoints.web.forwardedHeaders.insecure"
- "--entryPoints.web.proxyProtocol.insecure"
helm upgrade traefik2 -f values.yaml traefik-14.0.1.tgz -n kube-system
正式环境就先写的等维护的时候操作了!试过了没有问题:
traefik pod日志中可能会出现这样的日志:
参照:
https://doc.traefik.io/traefik/reference/dynamic-configuration/kubernetes-gateway/#rbac
cat gateway-rbac.yaml
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: gateway-role
rules:
- apiGroups:
- ""
resources:
- namespaces
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- services
- endpoints
- secrets
verbs:
- get
- list
- watch
- apiGroups:
- gateway.networking.k8s.io
resources:
- gatewayclasses
- gateways
- httproutes
- tcproutes
- tlsroutes
verbs:
- get
- list
- watch
- apiGroups:
- gateway.networking.k8s.io
resources:
- gatewayclasses/status
- gateways/status
- httproutes/status
- tcproutes/status
- tlsroutes/status
verbs:
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: gateway-controller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: gateway-role
subjects:
- kind: ServiceAccount
name: traefik-ingress-controller
namespace: traefik
kubectl apply -f gateway-rbac.yaml -n traefik