Istio的Ingress与Egress网关
一、认识Ingress
在Istio的流量控制中，Ingress可以理解为前端应用的一个代理网格，当被转发到代理容器的流量到达服务前，会先经过该服务的Ingress Gateway，之后；再有Ingress Gateway转发至后端的VirtualService资源对象上。
二、Ingress实验
1）部署httpbin服务
➜ cat ../../samples/httpbin/httpbin.yaml
# Copyright Istio Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
##################################################################################################
# httpbin service
##################################################################################################
apiVersion: v1
kind: ServiceAccount
metadata:
name: httpbin
---
apiVersion: v1
kind: Service
metadata:
name: httpbin
labels:
app: httpbin
service: httpbin
spec:
ports:
- name: http
port: 8000
targetPort: 80
selector:
app: httpbin
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: httpbin
spec:
replicas: 1
selector:
matchLabels:
app: httpbin
version: v1
template:
metadata:
labels:
app: httpbin
version: v1
spec:
serviceAccountName: httpbin
containers:
- image: docker.io/kennethreitz/httpbin
imagePullPolicy: IfNotPresent
name: httpbin
ports:
- containerPort: 80
➜ kubectl apply -f ../../samples/httpbin/httpbin.yaml
serviceaccount/httpbin created
service/httpbin created
deployment.apps/httpbin created
2）查看httpbin服务状态
➜ kubectl get pods -o wide | grep httpbin
httpbin-74fb669cc6-vn897 2/2 Running 0 4m15s 10.1.8.133 docker-desktop <none> <none>
➜ kubectl get rs -o wide | grep httpbin
httpbin-74fb669cc6 1 1 1 7m5s httpbin docker.io/kennethreitz/httpbin app=httpbin,pod-template-hash=74fb669cc6,version=v1
➜ kubectl get deploy -o wide | grep httpbin
httpbin 1/1 1 1 6m52s httpbin docker.io/kennethreitz/httpbin app=httpbin,version=v1
➜ kubectl get svc httpbin
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
httpbin ClusterIP 10.110.191.190 <none> 8000/TCP 12m
➜ kubectl describe svc httpbin
Name: httpbin
Namespace: default
Labels: app=httpbin
service=httpbin
Annotations: <none>
Selector: app=httpbin
Type: ClusterIP
IP: 10.110.191.190
Port: http 8000/TCP
TargetPort: 80/TCP
Endpoints: 10.1.8.133:80
Session Affinity: None
Events: <none>
➜ kubectl describe pods httpbin-74fb669cc6-vn897
Name: httpbin-74fb669cc6-vn897
Namespace: default
Priority: 0
Node: docker-desktop/192.168.65.4
Start Time: Sat, 03 Jul 2021 17:23:17 +0800
Labels: app=httpbin
istio.io/rev=default
pod-template-hash=74fb669cc6
security.istio.io/tlsMode=istio
service.istio.io/canonical-name=httpbin
service.istio.io/canonical-revision=v1
version=v1
Annotations: kubectl.kubernetes.io/default-container: httpbin
kubectl.kubernetes.io/default-logs-container: httpbin
prometheus.io/path: /stats/prometheus
prometheus.io/port: 15020
prometheus.io/scrape: true
sidecar.istio.io/status:
{"initContainers":["istio-init"],"containers":["istio-proxy"],"volumes":["istio-envoy","istio-data","istio-podinfo","istiod-ca-cert"],"ima...
Status: Running
IP: 10.1.8.133
IPs:
IP: 10.1.8.133
Controlled By: ReplicaSet/httpbin-74fb669cc6
Init Containers:
istio-init:
Container ID: docker://c2540ee240389633be39ef2c442f4b3a3bda1316a529d247a594c28bd1cefa32
Image: docker.io/istio/proxyv2:1.10.1
Image ID: docker-pullable://istio/proxyv2@sha256:d9b295da022ad826c54d5bb49f1f2b661826efd8c2672b2f61ddc2aedac78cfc
Port:
Host Port:
Args:
istio-iptables
-p
15001
-z
15006
-u
1337
-m
REDIRECT
-i
*
-x
-b
*
-d
15090,15021,15020
State: Terminated
Reason: Completed
Exit Code: 0
Started: Sat, 03 Jul 2021 17:23:21 +0800
Finished: Sat, 03 Jul 2021 17:23:21 +0800
Ready: True
Restart Count: 0
Limits:
cpu: 2
memory: 1Gi
Requests:
cpu: 10m
memory: 40Mi
Environment:
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from httpbin-token-qk94s (ro)
Containers:
httpbin:
Container ID: docker://3e4dfc72174241724b7c50c27d8d16947b43d126e435b916e39c5a925d79c1f8
Image: docker.io/kennethreitz/httpbin
Image ID: docker-pullable://kennethreitz/httpbin@sha256:599fe5e5073102dbb0ee3dbb65f049dab44fa9fc251f6835c9990f8fb196a72b
Port: 80/TCP
Host Port: 0/TCP
State: Running
Started: Sat, 03 Jul 2021 17:26:41 +0800
Ready: True
Restart Count: 0
Environment:
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from httpbin-token-qk94s (ro)
istio-proxy:
Container ID: docker://4c0a5a3b8bc0c7ed9f9fb5fac7690f5302652b76e5adf21dd58e4b6c6a8a5c8b
Image: docker.io/istio/proxyv2:1.10.1
Image ID: docker-pullable://istio/proxyv2@sha256:d9b295da022ad826c54d5bb49f1f2b661826efd8c2672b2f61ddc2aedac78cfc
Port: 15090/TCP
Host Port: 0/TCP
Args:
proxy
sidecar
--domain
$(POD_NAMESPACE).svc.cluster.local
--serviceCluster
httpbin.$(POD_NAMESPACE)
--proxyLogLevel=warning
--proxyComponentLogLevel=misc:error
--log_output_level=default:info
--concurrency
2
State: Running
Started: Sat, 03 Jul 2021 17:26:41 +0800
Ready: True
Restart Count: 0
Limits:
cpu: 2
memory: 1Gi
Requests:
cpu: 10m
memory: 40Mi
Readiness: http-get http://:15021/healthz/ready delay=1s timeout=3s period=2s #success=1 #failure=30
Environment:
JWT_POLICY: first-party-jwt
PILOT_CERT_PROVIDER: istiod
CA_ADDR: istiod.istio-system.svc:15012
POD_NAME: httpbin-74fb669cc6-vn897 (v1:metadata.name)
POD_NAMESPACE: default (v1:metadata.namespace)
INSTANCE_IP: (v1:status.podIP)
SERVICE_ACCOUNT: (v1:spec.serviceAccountName)
HOST_IP: (v1:status.hostIP)
CANONICAL_SERVICE: (v1:metadata.labels['service.istio.io/canonical-name'])
CANONICAL_REVISION: (v1:metadata.labels['service.istio.io/canonical-revision'])
PROXY_CONFIG: {}
ISTIO_META_POD_PORTS: [
{" containerPort":80,"protocol":"TCP"}
]
ISTIO_META_APP_CONTAINERS: httpbin
ISTIO_META_CLUSTER_ID: Kubernetes
ISTIO_META_INTERCEPTION_MODE: REDIRECT
ISTIO_META_WORKLOAD_NAME: httpbin
ISTIO_META_OWNER: kubernetes://apis/apps/v1/namespaces/default/deployments/httpbin
ISTIO_META_MESH_ID: cluster.local
TRUST_DOMAIN: cluster.local
Mounts:
/etc/istio/pod from istio-podinfo (rw)
/etc/istio/proxy from istio-envoy (rw)
/var/lib/istio/data from istio-data (rw)
/var/run/secrets/istio from istiod-ca-cert (rw)
/var/run/secrets/kubernetes.io/serviceaccount from httpbin-token-qk94s (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
istio-envoy:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium: Memory
SizeLimit:
istio-data:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit:
istio-podinfo:
Type: DownwardAPI (a volume populated by information about the pod)
Items:
metadata.labels -> labels
metadata.annotations -> annotations
limits.cpu -> cpu-limit
requests.cpu -> cpu-request
istiod-ca-cert:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: istio-ca-root-cert
Optional: false
httpbin-token-qk94s:
Type: Secret (a volume populated by a Secret)
SecretName: httpbin-token-qk94s
Optional: false
QoS Class: Burstable
Node-Selectors:
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 4m32s default-scheduler Successfully assigned default/httpbin-74fb669cc6-vn897 to docker-desktop
Normal Pulled 4m28s kubelet Container image " docker.io/istio/proxyv2:1.10.1" already present on machine
Normal Created 4m28s kubelet Created container istio-init
Normal Started 4m28s kubelet Started container istio-init
Normal Pulling 4m28s kubelet Pulling image "docker.io/kennethreitz/httpbin"
Normal Pulled 69s kubelet Successfully pulled image "docker.io/kennethreitz/httpbin" in 3m18.6849062s
Normal Created 69s kubelet Created container httpbin
Normal Started 68s kubelet Started container httpbin
Normal Pulled 68s kubelet Container image "docker.io/istio/proxyv2:1.10.1" already present on machine
Normal Created 68s kubelet Created container istio-proxy
Normal Started 68s kubelet Started container istio-proxy
3）创建Ingress Gateway
➜ cat ingress.yaml
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: httpbin-gateway
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "httpbin.example.com"
➜ kubectl apply -f ingress.yaml
gateway.networking.istio.io/httpbin-gateway created
➜ kubectl get gw httpbin-gateway -o wide
NAME AGE
httpbin-gateway 32s
➜ kubectl describe gw httpbin-gateway
Name: httpbin-gateway
Namespace: default
Labels: <none>
Annotations: <none>
API Version: networking.istio.io/v1beta1
Kind: Gateway
Metadata:
Creation Timestamp: 2021-07-03T09:32:32Z
Generation: 1
Managed Fields:
API Version: networking.istio.io/v1alpha3
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:kubectl.kubernetes.io/last-applied-configuration:
f:spec:
.:
f:selector:
.:
f:istio:
f:servers:
Manager: kubectl-client-side-apply
Operation: Update
Time: 2021-07-03T09:32:32Z
Resource Version: 632602
Self Link: /apis/networking.istio.io/v1beta1/namespaces/default/gateways/httpbin-gateway
UID: d1a475fb-a50a-485d-817c-4dedbb3426a9
Spec:
Selector:
Istio: ingressgateway
Servers:
Hosts:
httpbin.example.com
Port:
Name: http
Number: 80
Protocol: HTTP
Events: <none>
4）创建对应的Virtual Service
➜ cat virtualservice.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: httpbin
spec:
hosts:
- "httpbin.example.com"
gateways:
- httpbin-gateway
http:
- match:
- uri:
prefix: /status
- uri:
prefix: /delay
route:
- destination:
port:
number: 8000
host: httpbin
➜ kubectl apply -f virtualservice.yaml
virtualservice.networking.istio.io/httpbin created
5）查看Virtual Service状态
➜ kubectl get vs -o wide | grep httpbin
httpbin ["httpbin-gateway"] ["httpbin.example.com"] 56s
➜ kubectl describe vs httpbin
Name: httpbin
Namespace: default
Labels: <none>
Annotations: <none>
API Version: networking.istio.io/v1beta1
Kind: VirtualService
Metadata:
Creation Timestamp: 2021-07-03T09:39:26Z
Generation: 1
Managed Fields:
API Version: networking.istio.io/v1alpha3
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:kubectl.kubernetes.io/last-applied-configuration:
f:spec:
.:
f:gateways:
f:hosts:
f:http:
Manager: kubectl-client-side-apply
Operation: Update
Time: 2021-07-03T09:39:26Z
Resource Version: 633475
Self Link: /apis/networking.istio.io/v1beta1/namespaces/default/virtualservices/httpbin
UID: 4bf50ddd-6d1c-4e3b-948e-56d04b5f2332
Spec:
Gateways:
httpbin-gateway
Hosts:
httpbin.example.com
Http:
Match:
Uri:
Prefix: /status
Uri:
Prefix: /delay
Route:
Destination:
Host: httpbin
Port:
Number: 8000
Events: <none>
6）访问测试
httpbin服务有两个测试的url，分别是status与delay，status用于返回客户端传入的状态码，delay用于客户端自定义延迟多久返回响应报文。
➜ curl -I -HHost:httpbin.example.com http://localhost/status/200
HTTP/1.1 200 OK
server: istio-envoy
date: Sat, 03 Jul 2021 09:56:10 GMT
content-type: text/html; charset=utf-8
access-control-allow-origin: *
access-control-allow-credentials: true
content-length: 0
x-envoy-upstream-service-time: 29
➜ curl -I -HHost:httpbin.example.com http://localhost/status/301
HTTP/1.1 301 Moved Permanently
server: istio-envoy
date: Sat, 03 Jul 2021 09:57:08 GMT
location: /redirect/1
access-control-allow-origin: *
access-control-allow-credentials: true
content-length: 0
x-envoy-upstream-service-time: 44
➜ curl -I -HHost:httpbin.example.com http://localhost/status/502
HTTP/1.1 502 Bad Gateway
server: istio-envoy
date: Sat, 03 Jul 2021 09:56:38 GMT
content-type: text/html; charset=utf-8
access-control-allow-origin: *
access-control-allow-credentials: true
content-length: 0
x-envoy-upstream-service-time: 39
➜ curl -I -HHost:httpbin.example.com http://localhost/status/504
HTTP/1.1 504 Gateway Timeout
server: istio-envoy
date: Sat, 03 Jul 2021 09:56:52 GMT
content-type: text/html; charset=utf-8
access-control-allow-origin: *
access-control-allow-credentials: true
content-length: 0
x-envoy-upstream-service-time: 38
➜ time curl -I -HHost:httpbin.example.com http://localhost/delay/2
HTTP/1.1 200 OK
server: istio-envoy
date: Sat, 03 Jul 2021 09:57:35 GMT
content-type: application/json
content-length: 737
access-control-allow-origin: *
access-control-allow-credentials: true
x-envoy-upstream-service-time: 2009
curl -I -HHost:httpbin.example.com http://localhost/delay/2 0.00s user 0.01s system 0% cpu 2.023 total
➜ time curl -I -HHost:httpbin.example.com http://localhost/delay/5
HTTP/1.1 200 OK
server: istio-envoy
date: Sat, 03 Jul 2021 09:58:32 GMT
content-type: application/json
content-length: 737
access-control-allow-origin: *
access-control-allow-credentials: true
x-envoy-upstream-service-time: 5005
curl -I -HHost:httpbin.example.com http://localhost/delay/5 0.00s user 0.01s system 0% cpu 5.016 total
三、认识Egress
Egress网关允许您通过自定义Egress网关策略，来管理集群内部服务的流量出集群时的控制管理。
四、Egress实验
1）查看集群Egress Gateway组件是否存在
➜ kubectl get pods -n istio-system | grep egressgateway
istio-egressgateway-5796fb8ccf-jjtjl 1/1 Running 2 14d
2）创建sleep服务
➜ kubectl apply -f ../../samples/sleep/sleep.yaml
serviceaccount/sleep created
service/sleep created
deployment.apps/sleep created
➜ kubectl get pods -o wide | grep sleep
sleep-557747455f-rhj7k 2/2 Running 0 28s 10.1.8.134 docker-desktop <none> <none>
➜ kubectl get deploy -o wide | grep sleep
sleep 1/1 1 1 49s sleep curlimages/curl app=sleep
➜ kubectl get rs -o wide | grep sleep
sleep-557747455f 1 1 1 59s sleep curlimages/curl app=sleep,pod-template-hash=557747455f
➜ kubectl get svc -o wide | grep sleep
sleep ClusterIP 10.106.229.29 <none> 80/TCP 66s app=sleep
3）为外部服务定义Service Entry
➜ cat serviceentry.yaml
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: httpbin
spec:
hosts:
- httpbin.org
ports:
- number: 80
name: http-port
protocol: HTTP
resolution: DNS
➜ kubectl apply -f serviceentry.yaml
serviceentry.networking.istio.io/httpbin created
➜ kubectl get serviceentry -o wide
NAME HOSTS LOCATION RESOLUTION AGE
httpbin ["httpbin.org"] DNS 22s
➜ kubectl describe serviceentry httpbin
Name: httpbin
Namespace: default
Labels: <none>
Annotations: <none>
API Version: networking.istio.io/v1beta1
Kind: ServiceEntry
Metadata:
Creation Timestamp: 2021-07-03T10:11:12Z
Generation: 1
Managed Fields:
API Version: networking.istio.io/v1alpha3
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:kubectl.kubernetes.io/last-applied-configuration:
f:spec:
.:
f:hosts:
f:ports:
f:resolution:
Manager: kubectl-client-side-apply
Operation: Update
Time: 2021-07-03T10:11:12Z
Resource Version: 637531
Self Link: /apis/networking.istio.io/v1beta1/namespaces/default/serviceentries/httpbin
UID: ff8a43a4-a5d2-410c-ad7a-62d967f31c02
Spec:
Hosts:
httpbin.org
Ports:
Name: http-port
Number: 80
Protocol: HTTP
Resolution: DNS
Events: <none>
4）查看Egress Gateway日志，查看sleep服务访问外部服务是否经过Egress Gateway
➜ kubectl logs -f istio-egressgateway-5796fb8ccf-jjtjl -n istio-system
kubectl exec -it sleep-557747455f-rhj7k -c sleep -- curl http:/httpbin.org/ip
{
"origin": "58.49.32.170"
}
5）定义路由，将流量引导至Egress Gateway
➜ cat virtualservice.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: vs-for-egressgateway
spec:
hosts:
- httpbin.org
gateways:
- istio-egressgateway
- mesh
http:
- match:
- gateways:
- meth
port: 80
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.local
subset: httpbin
port:
number: 80
weight: 100
- match:
- gateways:
- istio-egressgateway
port: 80
route:
- destination:
host: httpbin.org
port:
number: 80
weight: 100
➜ kubectl apply -f virtualservice.yaml
virtualservice.networking.istio.io/vs-for-egressgateway created
➜ kubectl get vs -o wide | grep vs-for-egressgateway
vs-for-egressgateway ["istio-egressgateway","mesh"] ["httpbin.org"] 21s
➜ kubectl describe vs vs-for-egressgateway
Name: vs-for-egressgateway
Namespace: default
Labels: <none>
Annotations: <none>
API Version: networking.istio.io/v1beta1
Kind: VirtualService
Metadata:
Creation Timestamp: 2021-07-03T10:18:46Z
Generation: 1
Managed Fields:
API Version: networking.istio.io/v1alpha3
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:kubectl.kubernetes.io/last-applied-configuration:
f:spec:
.:
f:gateways:
f:hosts:
f:http:
Manager: kubectl-client-side-apply
Operation: Update
Time: 2021-07-03T10:18:46Z
Resource Version: 638484
Self Link: /apis/networking.istio.io/v1beta1/namespaces/default/virtualservices/vs-for-egressgateway
UID: cbd00a94-cb8d-46f5-9669-6df3584e0178
Spec:
Gateways:
istio-egressgateway
mesh
Hosts:
httpbin.org
Http:
Match:
Gateways:
meth
Port: 80
Route:
Destination:
Host: istio-egressgateway.istio-system.svc.cluster.local
Port:
Number: 80
Subset: httpbin
Weight: 100
Match:
Gateways:
istio-egressgateway
Port: 80
Route:
Destination:
Host: httpbin.org
Port:
Number: 80
Weight: 100
Events: <none>
➜ cat destinationrule.yaml
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: dr-for-egressgateway
spec:
host: istio-egressgateway.istio-system.svc.cluster.local
subsets:
- name: httpbin
➜ kubectl apply -f destinationrule.yaml
destinationrule.networking.istio.io/dr-for-egressgateway created
➜ kubectl get dr -o wide
NAME HOST AGE
dr-for-egressgateway istio-egressgateway.istio-system.svc.cluster.local 6s
➜ kubectl describe dr dr-for-egressgateway
Name: dr-for-egressgateway
Namespace: default
Labels: <none>
Annotations: <none>
API Version: networking.istio.io/v1beta1
Kind: DestinationRule
Metadata:
Creation Timestamp: 2021-07-03T10:21:31Z
Generation: 1
Managed Fields:
API Version: networking.istio.io/v1alpha3
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:kubectl.kubernetes.io/last-applied-configuration:
f:spec:
.:
f:host:
f:subsets:
Manager: kubectl-client-side-apply
Operation: Update
Time: 2021-07-03T10:21:31Z
Resource Version: 638834
Self Link: /apis/networking.istio.io/v1beta1/namespaces/default/destinationrules/dr-for-egressgateway
UID: bee9b39e-a9ac-4e36-803d-153bdf40c97d
Spec:
Host: istio-egressgateway.istio-system.svc.cluster.local
Subsets:
Name: httpbin
Events: <none>
6）再次使用sleep服务访问外部服务，查看Egress Gateway是否有相应的日志
kubectl exec -it sleep-557747455f-rhj7k -c sleep -- curl http:/httpbin.org/ip
{
"origin": "113.57.121.60"
}
➜ kubectl logs -l istio=egressgateway -c istio-proxy -n istio-system | tail
[2021-07-03 T18:38:49.810Z] "GET /politics HTTP/2" 200 - "-" "-" 0 0 1445 1444 "10.80.3.231" "curl/7.64.0" "2151bde2-4382-4e2f-b088-e464943c2a9b" "httpbin.org" "113.57.121.60:80" outbound|80||edition.cnn.com 10.80.3.232:51516 10.80.3.232:8080 10.80.3.231:38072 - -