k8s之Istio Gateway 单向/双向/透传TLS配置(1)

官网参考链接：https://istio.io/latest/zh/docs/tasks/traffic-management/ingress/secure-ingress/

流量转发过程

Gateway-单向TLS

• 外部流量到GATEWAY为HTTPS加密通信，GATEWAY到后端SERVICE的流量使用HTTP明文通信

准备证书

# 创建用于服务签名的根证书和私钥
	openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -subj '/O=example Inc./CN=example.com' -keyout example.com.key -out example.com.crt

# 为 httpbin.example.com 创建证书和私钥
	openssl req -out httpbin.example.com.csr -newkey rsa:2048 -nodes -keyout httpbin.example.com.key -subj "/CN=httpbin.example.com/O=httpbin organization"
	openssl x509 -req -sha256 -days 365 -CA example.com.crt -CAkey example.com.key -set_serial 0 -in httpbin.example.com.csr -out httpbin.example.com.crt

# 查看证书
[root@k8s-master-1 example-v1]# ls
example.com.crt  example.com.key  httpbin.example.com.crt  httpbin.example.com.csr  httpbin.example.com.key  

# 创建TLS secret，这个secret是否在istio-system命名空间都无所谓
	kubectl create -n istio-system secret tls httpbin-credential --key=httpbin.example.com.key --cert=httpbin.example.com.crt

准备测试

# 编写virutalservice
[root@k8s-master-1 example-v1]# cat istio.yaml 
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: httpbin-gateway
spec:
  selector:
    istio: ingressgateway # use istio default ingress gateway
  servers:
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: SIMPLE           # 单向TLS认证
      credentialName: httpbin-credential # must be the same as secret
    hosts:
    - "httpbin.example.com"
  - port: 
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "httpbin.example.com"
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: httpbin
spec:
  hosts:
  - "httpbin.example.com"
  gateways:
  - httpbin-gateway
  http:
  - route:
    - destination:
        port:
          number: 8000   # service端口
        host: "httpbin.default.svc.cluster.local"

# 部署istio自带的httpbin服务
[root@k8s-master-1 istio-1.12.6]# istioctl kube-inject -f samples/httpbin/httpbin.yaml | kubectl apply -f -
serviceaccount/httpbin created
service/httpbin created
deployment.apps/httpbin created

# 部署istio配置
[root@k8s-master-1 one-tls]# kubectl apply -f istio.yaml 
gateway.networking.istio.io/httpbin-gateway created
virtualservice.networking.istio.io/httpbin created

# 添加一个域名解析
	echo "192.168.0.10 httpbin.example.com" >> /etc/hosts

# 查看istio-gateway访问方式,由于我前面没有配置一个负载均衡进行流量转发，我这里访问暴露出来的端口了
[root@k8s-master-1 httpbin]# kubectl get svc -n istio-system                                              
istio-ingressgateway   LoadBalancer   10.0.83.120    <pending>     15021:38405/TCP,80:49967/TCP,443:49290/TCP,31400:38905/TCP,15443:36855/TCP   3d2h


# 访问测试，可见没有问题
[root@k8s-master-1 httpbin]# curl https://httpbin.example.com:49290/status/418 -k

    -=[ teapot ]=-

       _...._
     .'  _ _ `.
    | ."` ^ `". _,
    \_;`"---"`|//
      |       ;/
      \_     _/
        `"""`

# 指定CA公钥用来认证服务端发过来的证书
[root@k8s-master-1 example-v1]# curl https://httpbin.example.com:49290/status/418 --cacert example.com.crt 

    -=[ teapot ]=-

       _...._
     .'  _ _ `.
    | ."` ^ `". _,
    \_;`"---"`|//
      |       ;/
      \_     _/
        `"""`

# 访问测试：不忽略证书认证，由于颁发server端证书的CA机构curl无法使用系统内置CA公钥来验证服务端发过来的证书，故而无法正常建立连接
[root@k8s-master-1 httpbin]# curl -v https://httpbin.example.com:49290/status/418 
* About to connect() to httpbin.example.com port 49290 (#0)
*   Trying 192.168.0.10...
* Connected to httpbin.example.com (192.168.0.10) port 49290 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* Server certificate:
* 	subject: O=httpbin organization,CN=httpbin.example.com
* 	start date: Apr 27 03:37:45 2022 GMT
* 	expire date: Apr 27 03:37:45 2023 GMT
* 	common name: httpbin.example.com
* 	issuer: CN=example.com,O=example Inc.
* NSS error -8179 (SEC_ERROR_UNKNOWN_ISSUER)
* Peer's Certificate issuer is not recognized.
* Closing connection 0
curl: (60) Peer's Certificate issuer is not recognized.
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.

Gateway-双向TLS

• GATEWAY将会对客户端进行一次认证了，在客户端认证完服务端的证书后，将自身证书发送给GATEWAY网关

准备证书

# 删除之前的证书（之前创建的secret没有指定CA）
	kubectl -n istio-system delete secret httpbin-credential

# 创建认证相关的secret，这次加入了CA证书，用该CA来验证客户端发送过来的证书
	kubectl create -n istio-system secret generic httpbin-credential --from-file=tls.key=httpbin.example.com.key --from-file=tls.crt=httpbin.example.com.crt --from-file=ca.crt=example.com.crt

# 将单向TLS中的mode修改为：mode: MUTUAL，然后部署
[root@k8s-master-1 example-v1]# kubectl apply -f istio-two-tls.yaml 
gateway.networking.istio.io/httpbin-gateway configured
virtualservice.networking.istio.io/httpbin unchanged

准备测试

# 使用同一个CA生成客户端相关证书,这里一定要使用同个CA去颁发证书，否则会导致认证不了
	openssl req -out client.example.com.csr -newkey rsa:2048 -nodes -keyout client.example.com.key -subj "/CN=client.example.com/O=client organization"
	openssl x509 -req -sha256 -days 365 -CA example.com.crt -CAkey example.com.key -set_serial 1 -in client.example.com.csr -out client.example.com.crt

# 重新发送带客户端证书和私钥的 curl 请求。使用 –-cert 标志传递客户端证书，使用 -–key 标志传递私钥（发送数据的时候会使用私钥加密数据）
[root@k8s-master-1 example-v1]# curl -v --cacert ./example.com.crt --cert ./client.example.com.crt --key ./client.example.com.key https://httpbin.example.com:49290/status/418
* About to connect() to httpbin.example.com port 49290 (#0)
*   Trying 192.168.0.10...
* Connected to httpbin.example.com (192.168.0.10) port 49290 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: ./example.com.crt
  CApath: none
* NSS: client certificate from file
* 	subject: O=client organization,CN=client.example.com
* 	start date: Apr 27 04:57:49 2022 GMT
* 	expire date: Apr 27 04:57:49 2023 GMT
* 	common name: client.example.com
* 	issuer: CN=example.com,O=example Inc.
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
* 	subject: O=httpbin organization,CN=httpbin.example.com
* 	start date: Apr 27 03:37:45 2022 GMT
* 	expire date: Apr 27 03:37:45 2023 GMT
* 	common name: httpbin.example.com
* 	issuer: CN=example.com,O=example Inc.
> GET /status/418 HTTP/1.1
> User-Agent: curl/7.29.0
> Host: httpbin.example.com:49290
> Accept: */*
> 
< HTTP/1.1 418 Unknown
< server: istio-envoy
< date: Wed, 27 Apr 2022 06:04:52 GMT
< x-more-info: http://tools.ietf.org/html/rfc2324
< access-control-allow-origin: *
< access-control-allow-credentials: true
< content-length: 135
< x-envoy-upstream-service-time: 8
< 

    -=[ teapot ]=-

       _...._
     .'  _ _ `.
    | ."` ^ `". _,
    \_;`"---"`|//
      |       ;/
      \_     _/
        `"""`
* Connection #0 to host httpbin.example.com left intact


# 不发送给服务端证书，且不验证服务端证书访问，可以发现客户端如果不发送证书给服务端验证，会无法正常访问
[root@k8s-master-1 example-v1]# curl -v -k https://httpbin.example.com:49290/status/418
* About to connect() to httpbin.example.com port 49290 (#0)
*   Trying 192.168.0.10...
* Connected to httpbin.example.com (192.168.0.10) port 49290 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* NSS: client certificate not found (nickname not specified)
* NSS error -12227 (SSL_ERROR_HANDSHAKE_FAILURE_ALERT)
* SSL peer was unable to negotiate an acceptable set of security parameters.
* Closing connection 0
curl: (35) NSS: client certificate not found (nickname not specified)

Gateway-透传TLS

• 外部流量到GATEWAY不会进行认证了，且GTATEWAY到service的流量也不做是否设置HTTPS配置，后端有HTTPS，GATEWAY就直接传过去，该模式适合后端自带HTTPS认证的

准备证书

# 创建根证书和私钥来为您的服务签名证书：
	openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -subj '/O=example Inc./CN=example.com' -keyout example.com.key -out example.com.crt

# 为 nginx.example.com 创建证书和私钥
	openssl req -out nginx.example.com.csr -newkey rsa:2048 -nodes -keyout nginx.example.com.key -subj "/CN=nginx.example.com/O=some organization"
	openssl x509 -req -sha256 -days 365 -CA example.com.crt -CAkey example.com.key -set_serial 0 -in nginx.example.com.csr -out nginx.example.com.crt

# 创建一个 Kubernetes 的 Secret 资源来保存服务的证书
	kubectl create secret tls nginx-server-certs --key nginx.example.com.key --cert nginx.example.com.crt

部署NGINX 服务

# 为 NGINX 服务创建一个配置文件：
cat <<\EOF > ./nginx.conf
events {
}

http {
  log_format main '$remote_addr - $remote_user [$time_local]  $status '
  '"$request" $body_bytes_sent "$http_referer" '
  '"$http_user_agent" "$http_x_forwarded_for"';
  access_log /var/log/nginx/access.log main;
  error_log  /var/log/nginx/error.log;

  server {
    listen 443 ssl;

    root /usr/share/nginx/html;
    index index.html;

    server_name nginx.example.com;
    ssl_certificate /etc/nginx-server-certs/tls.crt;
    ssl_certificate_key /etc/nginx-server-certs/tls.key;
  }
}
EOF

# 创建一个 Kubernetes 的 ConfigMap 资源来保存 NGINX 服务的配置
	kubectl create configmap nginx-configmap --from-file=nginx.conf=./nginx.conf

# 部署 NGINX 服务
cat <| istioctl kube-inject -f - | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
  name: my-nginx
  labels:
    run: my-nginx
spec:
  ports:
  - port: 443
    protocol: TCP
  selector:
    run: my-nginx
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-nginx
spec:
  selector:
    matchLabels:
      run: my-nginx
  replicas: 1
  template:
    metadata:
      labels:
        run: my-nginx
    spec:
      containers:
      - name: my-nginx
        image: nginx
        ports:
        - containerPort: 443
        volumeMounts:
        - name: nginx-config
          mountPath: /etc/nginx
          readOnly: true
        - name: nginx-server-certs
          mountPath: /etc/nginx-server-certs
          readOnly: true
      volumes:
      - name: nginx-config
        configMap:
          name: nginx-configmap
      - name: nginx-server-certs
        secret:
          secretName: nginx-server-certs
EOF

访问测试

# 访问nginx svc，可见能正常访问
[root@k8s-master-1 passthrough-tls]# kubectl exec "$(kubectl get pod  -l run=my-nginx -o jsonpath={.items..metadata.name})" -c istio-proxy -- curl -I -k -s --resolve nginx.example.com:443:127.0.0.1 https://nginx.example.com
HTTP/1.1 200 OK
Server: nginx/1.21.5
Date: Wed, 27 Apr 2022 06:51:17 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Tue, 28 Dec 2021 15:28:38 GMT
Connection: keep-alive
ETag: "61cb2d26-267"
Accept-Ranges: bytes

配置 Ingress Gateway

定义一个 server 部分的端口为 443 的 Gateway。注意，PASSTHROUGH tls TLS 模式，该模式指示 Gateway 以 AS IS 方式传递入口流量，而不终止 TLS

kubectl apply -f - <apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: mygateway
spec:
  selector:
    istio: ingressgateway # use istio default ingress gateway
  servers:
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: PASSTHROUGH
    hosts:
    - nginx.example.com
EOF


# 为通过 Gateway 进入的流量配置路由：
kubectl apply -f - <apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx
spec:
  hosts:
  - nginx.example.com
  gateways:
  - mygateway
  tls:
  - match:
    - port: 443
      sniHosts:
      - nginx.example.com
    route:
    - destination:
        host: my-nginx
        port:
          number: 443
EOF

# 查看svc
[root@k8s-master-1 passthrough-tls]# kubectl get svc -n istio-system
NAME                   TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)                                               
istio-egressgateway    ClusterIP      10.0.253.161   <none>        80/TCP,443/TCP                                      
istio-ingressgateway   LoadBalancer   10.0.83.120    <pending>     15021:38405/TCP,80:49967/TCP,443:49290/TCP,31400:38905/TCP,15443:36855/TCP

# 访问测试
[root@k8s-master-1 passthrough-tls]# curl -I --resolve "nginx.example.com:49290:192.168.0.10" --cacert example.com.crt "https://nginx.example.com:49290"
HTTP/1.1 200 OK
Server: nginx/1.21.5
Date: Wed, 27 Apr 2022 06:49:10 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Tue, 28 Dec 2021 15:28:38 GMT
Connection: keep-alive
ETag: "61cb2d26-267"
Accept-Ranges: bytes

# 不指定CA公钥认证服务器发送过来的证书，可见无法正常访问
[root@k8s-master-1 passthrough-tls]# curl -I --resolve "nginx.example.com:49290:192.168.0.10" "https://nginx.example.com:49290"
curl: (60) Peer's Certificate issuer is not recognized.
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.

清除环境

# 删除已创建的 Kubernetes 资源
    kubectl delete secret nginx-server-certs
    kubectl delete configmap nginx-configmap
    kubectl delete service my-nginx
    kubectl delete deployment my-nginx
    kubectl delete gateway mygateway
    kubectl delete virtualservice nginx

# 删除证书和密钥
	rm example.com.crt example.com.key nginx.example.com.crt nginx.example.com.key nginx.example.com.csr

# 删除本示例中生成的配置文件
	 rm ./nginx.conf