Kubernetes 服务入口管理 Traefik Ingress Controller

前面部署了 kubernetes/ingress-nginx 作为 Ingress Controller，使用 Nginx 反向代理与负载，通过 Ingress Controller 不断的跟 Kubernetes API 交互，实时获取后端 Service、Pod 等的变化，然后动态更新 Nginx 配置，并刷新使配置生效。Traefik 是一个用 Golang 开发的轻量级的 Http 反向代理和负载均衡器软件，由于可以自动化配置和刷新 backend 节点，目前可以被绝大部分容器平台与组件支持，例如 Docker, Swarm mode, Kubernetes,, Consul, Etcd, Rancher, Eureka 等。Traefik 设计的就能够实时跟 Kubernetes API 交互，感知后端 Service、Pod 等的变化，自动更新配置并热重载，使用上大体上差不多，但是 Traefik 更快速更方便，同时支持更多的特性，使反向代理、负载均衡更直接，更高效。

Traefik 特性

• 自动化动态配置无需服务重启
• 支持多个负载平衡算法
• 支持 Let’s Encrypt (通配符支持) 向您的微服务提供 HTTPS
• 支持熔断，重试
• 集群模式的高可用性
• 提供简洁的 UI 界面
• 支持 Websocket, HTTP/2, GRPC 协议
• 提供监控的服务(Rest、Prometheus、Datadog、Statsd、InfluxDB)
• 保留访问日志（JSON，CLF）
• 快速
• 支持 Rest API
• 使用二进制文件打包，并作为一个 docker 镜像提供

部署 Traefik

所有的配置文件可以在官方的 github 仓库中找到,按照官方文档来即可。

Role Based Access Control configuration (Kubernetes 1.6+ only)

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
  name: traefik-ingress-controller
  namespace: kube-system

$ kubectl apply -f https://raw.githubusercontent.com/containous/traefik/master/examples/k8s/traefik-rbac.yaml

Deploy Traefik using a Deployment or DaemonSet

DaemonSet 会在每台 Node 节点上都创建 Pod 而 Deployment  是人为控制的副本数量（根据实际需求来取决），这里使用 DaemonSet 类型来部署 Traefik。

部署 Traefik（修改 hostNetwork: true）

#https://raw.githubusercontent.com/containous/traefik/master/examples/k8s/traefik-ds.yamlvi traefik-ds.yaml

vi traefik-ds.yaml

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
---
kind: DaemonSet
apiVersion: extensions/v1beta1
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
  labels:
    k8s-app: traefik-ingress-lb
spec:
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      hostNetwork: true
      restartPolicy: Always
      containers:
      - image: traefik
        name: traefik-ingress-lb
        ports:
        - name: http
          containerPort: 80
          hostPort: 80
        - name: admin
          containerPort: 8080
          hostPort: 8080
        securityContext:
          capabilities:
            drop:
            - ALL
            add:
            - NET_BIND_SERVICE
        args:
        - --api
        - --kubernetes
        - --logLevel=INFO
---
kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-service
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
    - protocol: TCP
      port: 80
      name: web
    - protocol: TCP
      port: 8080
      name: admin

$ kubectl apply -f traefik-ds.yaml
serviceaccount/traefik-ingress-controller unchanged
daemonset.extensions/traefik-ingress-controller configured
service/traefik-ingress-service unchanged
$ kubectl apply -f traefik-ds.yaml
serviceaccount/traefik-ingress-controller unchanged
daemonset.extensions/traefik-ingress-controller unchanged
service/traefik-ingress-service unchanged
[root@kubernetes-master k8s]# kubectl -n kube-system get pod -o wide
NAME                                        READY   STATUS    RESTARTS   AGE   IP              NODE                NOMINATED NODE
traefik-ingress-controller-6fk9n            1/1     Running   0          41m   10.38.0.0       kubernetes-node-2   
traefik-ingress-controller-f7kmc            1/1     Running   0          41m   10.40.0.1       kubernetes-node-1   

备注：上述由于修改 hostNetwork: true ，其实已经在每个 Node 节点开放了 80 与 8080 端口，80 提供正常服务，8080 是其自带的 UI 界面。

Node 节点查看开放的端口

$  netstat -ntlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name      
tcp6       0      0 :::8080                 :::*                    LISTEN      10253/traefik       
tcp6       0      0 :::80                   :::*                    LISTEN      10253/traefik

Ingress 方式暴露 Traefik Web UI

#  https://raw.githubusercontent.com/containous/traefik/master/examples/k8s/ui.yaml
vi traefik-web-ui.yaml

---
apiVersion: v1
kind: Service
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
  - name: web
    port: 80
    targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: kube-system
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  rules:
  - host: traefik-ui.com
    http:
      paths:
      - backend:
          serviceName: traefik-web-ui
          servicePort: 80

部署查看

$ kubectl apply -f traefik-web-ui.yaml
service/traefik-web-ui created
ingress.extensions/traefik-web-ui created
$ kubectl get ingress -o wide --all-namespaces   
NAMESPACE     NAME             HOSTS            ADDRESS   PORTS   AGE
kube-system   traefik-web-ui   traefik-ui.com             80      18s

配置Host文件

172.23.216.49 k8s.dashboard.com
172.23.216.49 traefik-ui.com

访问 http://traefik-ui.com/dashboard/ 通过 80 端口转发。

模拟部署一个程序

下面模拟部署一个程序，已 Nginx  为例：

vi nginx-deployment.yaml

apiVersion: v1
kind: Service
metadata:
  name: nginx-svc
spec:
  template:
    metadata:
      labels:
        name: nginx-svc
        namespace: default
spec:
  selector:
    run: nginx-pod
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: nginx-pod
spec:
  replicas: 4
  template:
    metadata:
      labels:
        run: nginx-pod
    spec:
      containers:
      - name: nginx
        image: nginx:1.15.5
        ports:
        - containerPort: 80
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ngx-ing
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  rules:
  - host: k8s.nginx.com
    http:
      paths:
      - backend:
          serviceName: nginx-svc
          servicePort: 80

部署查看

$ kubectl apply -f nginx-deployment.yam
$  kubectl get pods  --all-namespaces
NAMESPACE     NAME                                        READY   STATUS    RESTARTS   AGE
default       nginx-pod-5b5bc94455-ndcl6                  1/1     Running   0          18m
default       nginx-pod-5b5bc94455-nptm5                  1/1     Running   0          18m
default       nginx-pod-5b5bc94455-ptvzp                  1/1     Running   0          18m
default       nginx-pod-5b5bc94455-vw667                  1/1     Running   0          18m

修改 Host 文件

172.23.216.49 k8s.dashboard.com
172.23.216.49 traefik-ui.com
172.23.216.49 k8s.nginx.com

访问 k8s.nginx.com 即可，查看 traefik-ui(对应 4个 Pod)。

HTTPS 证书配置

生成自签名证书

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=traefik-ui.com"
kubectl -n kube-system create secret tls traefik-ui-tls-cert --key=tls.key --cert=tls.crt

配置

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: kube-system
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  rules:
  - host: traefik-ui.com
    http:
      paths:
      - backend:
          serviceName: traefik-web-ui
          servicePort: 80
  tls:
   - secretName: traefik-ui-tls-cert

其他问题：

Basic Authentication

如果想让 traefik-web-ui 需要验证才能访问，可以配置 Basic Authentication，具体看官方文档。

https://docs.traefik.io/user-guide/kubernetes/#basic-authentication

GRpc 配置

https://docs.traefik.io/user-guide/grpc/

服务权重配置

https://docs.traefik.io/user-guide/kubernetes/#traffic-splitting

证书配置

https://docs.traefik.io/user-guide/examples/#http-redirect-on-https

REFER:
https://docs.traefik.io/user-guide/kubernetes/
https://github.com/containous/traefik/tree/master/examples/k8s
http://blog.leanote.com/post/criss_sun/K8S%E4%B9%8Btraefik
http://blog.51cto.com/ylw6006/2119784