istio-不能自动注入sidecar的解决方法
参考：https://github.com/opsnull/follow-me-install-kubernetes-cluster 使用二进制文件安装kubernetes,安装istio后，istio 不能自动注入sidecar.
kubernetes 1.9 +
istio 1.0
以下是解决方法：
1. kube-apiserver 增加 adminsion（MutatingAdmissionWebhook,ValidatingAdmissionWebhook）
编辑 /etc/systemd/system/kube-apiserver.service
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target
[Service]
ExecStart=/opt/k8s/bin/kube-apiserver \
--enable-admission-plugins=Initializers,NamespaceLifecycle,NodeRestriction,LimitRanger,ServiceAccount,DefaultStorageClass,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota
2. master节点没安装 flanneld、docker、kube-proxy :
会造成master节点访问不了集群内部的pod(istio-sidecar-injector-5b5fcf4df6-vj8fl)，导致自动注入失败。
[k8s@k8s-01 metrics-server]$ kubectl describe replicaset.apps/nginx-75675f5897
Name: nginx-75675f5897
Namespace: default
Selector: app=nginx,pod-template-hash=3123191453
Labels: app=nginx
pod-template-hash=3123191453
Annotations: deployment.kubernetes.io/desired-replicas=1
deployment.kubernetes.io/max-replicas=2
deployment.kubernetes.io/revision=1
Controlled By: Deployment/nginx
Replicas: 0 current / 1 desired
Pods Status: 0 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
Labels: app=nginx
pod-template-hash=3123191453
Containers:
nginx:
Image: nginx:1.7.9
Port: 80/TCP
Host Port: 0/TCP
Environment:
Mounts:
Volumes:
Conditions:
Type Status Reason
---- ------ ------
ReplicaFailure True FailedCreate
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedCreate 10m replicaset-controller Error creating: Internal error occurred: failed calling admission webhook "sidecar-injector.istio.io": Post https://istio-sidecar-injector.istio-system.svc:443/inject?timeout=30s: dial tcp 172.30.17.12:443: getsockopt: network is unreachable
Warning FailedCreate 4m (x16 over 16m) replicaset-controller Error creating: Internal error occurred: failed calling admission webhook "sidecar-injector.istio.io": Post https://istio-sidecar-injector.istio-system.svc:443/inject?timeout=30s: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
解决方法：
在master节点安装：flanneld、docker、kube-proxy
3. 没有配置Aggregation（最好安装 metrics-server 组件，并测试是否可以正常运行和能够获取metrics,并试用hpa是否成功） ：
$ systemctl status kube-controller-manager
7月 29 22:13:58 k8s-01 kube-controller-manager[4318]: I0729 22:13:58.554076 4318 event.go:218] Event(v1.ObjectReference{Kind:"HorizontalPodAutoscaler", Namespace:"istio-system", Name:"istio-ingressgateway", UID:"353e14b3-9391-11e8-b248-08002768bc4c", APIVersion:"autoscaling/v2beta1", ResourceVersion:"3432740", FieldPath:""}): type: 'Warning' reason: 'FailedComputeMetricsReplicas' failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server could not find the requested resource (get pods.metrics.k8s.io)
$ kubectl describe replicaset.apps/nginx-75675f5897
Name: nginx-75675f5897
Namespace: default
Selector: app=nginx,pod-template-hash=3123191453
Labels: app=nginx
pod-template-hash=3123191453
Annotations: deployment.kubernetes.io/desired-replicas=1
deployment.kubernetes.io/max-replicas=2
deployment.kubernetes.io/revision=1
Controlled By: Deployment/nginx
Replicas: 0 current / 1 desired
Pods Status: 0 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
Labels: app=nginx
pod-template-hash=3123191453
Containers:
nginx:
Image: nginx:1.7.9
Port: 80/TCP
Host Port: 0/TCP
Environment:
Mounts:
Volumes:
Conditions:
Type Status Reason
---- ------ ------
ReplicaFailure True FailedCreate
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedCreate 29s replicaset-controller Error creating: Internal error occurred: failed calling admission webhook "sidecar-injector.istio.io": Post https://istio-sidecar-injector.istio-system.svc:443/inject?timeout=30s: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
解决方法：
参考 ：
https://zhuanlan.zhihu.com/p/27723597
https://github.com/opsnull/follow-me-install-kubernetes-cluster/blob/master/09-4.metrics-server%E6%8F%92%E4%BB%B6.md
API Aggregation允许在不修改Kubernetes核心代码的同时扩展Kubernetes API。
增加下面参数：
kube-controller-manager ：
--horizontal-pod-autoscaler-use-rest-clients=true
kube-apiserver:
--requestheader-client-ca-file=/etc/kubernetes/cert/ca.pem \
--requestheader-allowed-names= \
--requestheader-extra-headers-prefix="X-Remote-Extra-" \
--requestheader-group-headers=X-Remote-Group \
--requestheader-username-headers=X-Remote-User \
--proxy-client-cert-file=/etc/kubernetes/cert/metrics-server.pem \
--proxy-client-key-file=/etc/kubernetes/cert/metrics-server-key.pem \
--runtime-config=api/all=true \
--enable-aggregator-routing=true
4. 创建pod，状态一直是Init:Blocked 问题：
$ kubectl get all
NAME READY STATUS RESTARTS AGE
pod/nginx-75675f5897-5srkl 0/2 Init:Blocked 0 53m
$ systemctl status kubelet -l
7月 31 21:46:46 k8s-03 kubelet[1588]: E0731 21:46:46.264170 1588 pod_workers.go:186] Error syncing pod 2f51514f-952b-11e8-94fb-08002768bc4c ("nginx-75675f5897-wskvw_default(2f51514f-952b-11e8-94fb-08002768bc4c)"), skipping: pod cannot be run: pod with UID "2f51514f-952b-11e8-94fb-08002768bc4c" specified privileged init container, but is disallowed
解决方法：
在 /etc/systemd/system/kubelet.service 增加启动参数：
--allow-privileged=true