Kubernetes-1.16部署之三 Traefik 2.0
一、创建Traefik CRD资源
traefik v2.0 版本后，开始使用CRD（Custom Resource Definition）来完成路由配置
# mkdir -p /opt/kubernetes/traefik/yaml
# cd /opt/kubernetes/traefik/yaml
# vi crd.yaml
## IngressRoute
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: ingressroutes.traefik.containo.us
spec:
group: traefik.containo.us
version: v1alpha1
names:
kind: IngressRoute
plural: ingressroutes
singular: ingressroute
scope: Namespaced
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: ingressroutetcps.traefik.containo.us
spec:
group: traefik.containo.us
version: v1alpha1
names:
kind: IngressRouteTCP
plural: ingressroutetcps
singular: ingressroutetcp
scope: Namespaced
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: middlewares.traefik.containo.us
spec:
group: traefik.containo.us
version: v1alpha1
names:
kind: Middleware
plural: middlewares
singular: middleware
scope: Namespaced
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: tlsoptions.traefik.containo.us
spec:
group: traefik.containo.us
version: v1alpha1
names:
kind: TLSOption
plural: tlsoptions
singular: tlsoption
scope: Namespaced创建Rraefik CRD资源
# kubectl create -f crd.yaml
customresourcedefinition.apiextensions.k8s.io/ingressroutes.traefik.containo.us created
customresourcedefinition.apiextensions.k8s.io/ingressroutetcps.traefik.containo.us created
customresourcedefinition.apiextensions.k8s.io/middlewares.traefik.containo.us created
customresourcedefinition.apiextensions.k8s.io/tlsoptions.traefik.containo.us created二、创建Traefik RBAC权限
# vi rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: traefik-ingress-controller
namespace: kube-system
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: traefik-ingress-controller
rules:
- apiGroups:
- ""
resources:
- services
- endpoints
- secrets
verbs:
- get
- list
- watch
- apiGroups:
- extensions
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- extensions
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- traefik.containo.us
resources:
- middlewares
verbs:
- get
- list
- watch
- apiGroups:
- traefik.containo.us
resources:
- ingressroutes
verbs:
- get
- list
- watch
- apiGroups:
- traefik.containo.us
resources:
- ingressroutetcps
verbs:
- get
- list
- watch
- apiGroups:
- traefik.containo.us
resources:
- tlsoptions
verbs:
- get
- list
- watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: traefik-ingress-controller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
name: traefik-ingress-controller
namespace: kube-system创建Traefik RBAC资源
# kubectl create -f rbac.yaml
serviceaccount/traefik-ingress-controller created
clusterrole.rbac.authorization.k8s.io/traefik-ingress-controller created
clusterrolebinding.rbac.authorization.k8s.io/traefik-ingress-controller created三、创建Traefik配置文件
用DaemonSet方式部署，便于在多服务器间扩展
# vi traefik.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: traefik
namespace: kube-system
labels:
k8s-app: traefik-ingress-lb
spec:
selector:
matchLabels:
k8s-app: traefik-ingress-lb
template:
metadata:
labels:
k8s-app: traefik-ingress-lb
name: traefik-ingress-lb
spec:
serviceAccountName: traefik-ingress-controller
terminationGracePeriodSeconds: 60
restartPolicy: Always
tolerations:
- operator: "Exists"
containers:
- image: traefik:v2.0.7
name: traefik-ingress-lb
resources:
limits:
cpu: 2000m
memory: 1024Mi
requests:
cpu: 1000m
memory: 1024Mi
ports:
- name: web
containerPort: 80
hostPort: 80
- name: websecure
containerPort: 443
hostPort: 443
- name: mysql
containerPort: 3306
hostPort: 3306
- name: redis
containerPort: 6379
hostPort: 6379
- name: admin
containerPort: 8080
hostPort: 9999
securityContext:
capabilities:
drop:
- ALL
add:
- NET_BIND_SERVICE
args:
- --entrypoints.web.Address=:80
- --entrypoints.websecure.Address=:443
- --entrypoints.mysql.Address=:3306
- --entrypoints.redis.Address=:6379
- --providers.kubernetescrd
- --api
- --api.dashboard=true
- --api.insecure=true
- --metrics.prometheus=true
- --tracing.zipkin=true
- --accesslog
- --accesslog.filepath=/var/log/access.log
nodeSelector:
edgenode: "true"
---
kind: Service
apiVersion: v1
metadata:
name: traefik
namespace: kube-system
spec:
selector:
k8s-app: traefik-ingress-lb
ports:
- name: admin
port: 9999
protocol: TCP创建Traefik资源
# kubectl create -f traefik.yaml
daemonset.apps/traefik created
service/traefik created四、设置节点Label标签
由于是使用Kubernetes DeamonSet这种方式部署traefik，所以需要提前给节点设置label，这样当程序部署时pod会自动调度到设置label的点上
# kubectl get nodes --show-labels
NAME STATUS ROLES AGE VERSION LABELS
192.168.168.3 Ready 4d11h v1.16.2 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=192.168.168.3,kubernetes.io/os=linux
192.168.168.4 Ready 4d11h v1.16.2 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=192.168.168.4,kubernetes.io/os=linux
# kubectl label nodes 192.168.168.3 edgenode=true
node/192.168.168.3 labeled 注：如想删除label标签使用如下命令
# kubectl label nodes 192.168.168.3 edgenode-五、配置Traefik路由规则
想让外部访问Kubernetes内部服务，需要配置路由规则，这里配置Traefik Dashboard的路由规则，使外部能够访问Traefik Dashboard
# vi IngressRoute.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: traefik-webui
namespace: kube-system
spec:
entryPoints:
- web
routes:
- match: Host(`traefik.k8s.local`)
kind: Rule
services:
- name: traefik
port: 9999注:Host(` `)中的内容也可为自定义域名，如配置为traefik.
创建Traefik Dashboard https协议路由规则对象
# kubectl create -f IngressRoute.yaml
ingressroute.traefik.containo.us/traefik-webui created注：卸载Traefik时先卸载IngressRoute.yaml再卸载其它资源
在管理机hosts里配置映射192.168.168.3 traefik.k8s.local(或搭建内网DNS服务器)，然后在浏览器中输入http://traefik.k8s.local:9999
六、配置多边缘节点高可用
1）在work-node01/02上安装keepalived｛安装过程此文略过｝
设置一个VIP IP指向自定义域名traefik.k8s.local，本文示例为192.168.168.100，这样集群外部就可以通过service的DNS映射名称来访问服务。
2）设置节点Label标签
# kubectl label nodes 192.168.168.3 edgenode=true
# kubectl label nodes 192.168.168.4 edgenode=true3）查看DaemonSet启动情况
# kubectl -n kube-system get ds
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
traefik 2 2 2 2 2 edgenode=true 16m4）配置keepalived
# vi $KEEPALIVED_HOME/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from k8s_admin@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 6666
}
virtual_ipaddress {
192.168.168.100
}
}
virtual_server 192.168.168.100 9999{
delay_loop 6
lb_algo loadbalance
lb_kind DR
nat_mask 255.255.255.0
persistence_timeout 0
protocol TCP
real_server 192.168.168.3 9999{
weight 1
TCP_CHECK {
connect_timeout 3
}
}
real_server 192.168.168.4 9999{
weight 1
TCP_CHECK {
connect_timeout 3
}
}
}注：real_server的IP和端口即traefik供外网访问的IP和端口
在管理机hosts里配置映射192.168.168.100 traefik.k8s.local，然后在在浏览器中输入http://traefik.k8s.local