K8S+Cloud-Controller-Manager对接Openstack Cinder创建PV之三:使用local-up-cluster.sh创建本地K8S集群存在的问题

说明

使用local-up-cluster.sh创建的集群，主要是为了验证cloud-controller-manager（CCM）功能
本地集群创建过程可参考local-up-cluster.sh创建集群连接openstack

1、问题1：此集群就是测试demo无法用于生产环境

从打印日志来看，此集群在shell前台运行，如果退出shell，则集群就会销毁

Local Kubernetes cluster is running. Press Ctrl-C to shut it down.

注：集群销毁后，openstack-cloud-controller-manager进程还在，这应该是个bug

4 S root      7506     1  0  80   0 - 36287 ep_pol Jan20 ?        00:08:07 /home/k8s/linux-amd64/openstack-cloud-controller-manager --v=3 --vmodule=  --feature-gates=AllAlpha=false --cloud-provider=openstack --cloud-config=/etc/kubernetes/cloud-config --kubeconfig /var/run/kubernetes/controller.kubeconfig --use-service-account-credentials --leader-elect=false --master=https://localhost:6443

2、问题2：pod kube-dns-6c87f7688f-rlgsz处于pending状态

原因是，对于应用ccm组件的k8s集群，node节点会打上一个uninitialized=true:NoSchedule的taint，而dns pod无法容忍改taint，所以调度失败

# ./cluster/kubectl.sh get pods -n kube-system
NAME                        READY   STATUS    RESTARTS   AGE
kube-dns-6c87f7688f-rlgsz   0/3     Pending   0          11m

# ./cluster/kubectl.sh describe pods -n kube-system
...
Events:
  Type     Reason            Age        From               Message
  ----     ------            ----       ----               -------
  Warning  FailedScheduling    default-scheduler  0/1 nodes are available: 1 node(s) had taints that the pod didn't tolerate.
  Warning  FailedScheduling    default-scheduler  0/1 nodes are available: 1 node(s) had taints that the pod didn't tolerate.

# ./cluster/kubectl.sh get nodes
NAME        STATUS   ROLES    AGE   VERSION
127.0.0.1   Ready       12m   v1.17.0

# ./cluster/kubectl.sh describe nodes 127.0.0.1
...
Taints:             node.cloudprovider.kubernetes.io/uninitialized=true:NoSchedule

这进一步印证了问题1的观点

3、问题3：易用性较差

比如：
无法配置本地镜像仓库
只能跑在shell前台
…

4、问题4：使用了read-only port，默认为10255

这严格说不算个问题，10255是一个非认证端口，一般情况下会禁用，只有测试环境会打开

The read-only port for the Kubelet to serve on with no authentication/authorization (set to 0 to disable) (default 10255) 

但是在同一个节点如果需要运行开源的virtual-kubelet时，会报端口冲突

# ./virtual-kubelet --provider openstack --kubeconfig /var/run/kubernetes/kubelet.kubeconfig
E0119 06:28:57.156938    6439 reflector.go:126] pkg/mod/k8s.io/[email protected]+incompatible/tools/cache/reflector.go:94: Failed to list *v1.Secret: secrets is forbidden: User "system:node:127.0.0.1" cannot list resource "secrets" in API group "" at the cluster scope: can only read namespaced object of this type
E0119 06:28:57.157109    6439 reflector.go:126] pkg/mod/k8s.io/[email protected]+incompatible/tools/cache/reflector.go:94: Failed to list *v1.ConfigMap: configmaps is forbidden: User "system:node:127.0.0.1" cannot list resource "configmaps" in API group "" at the cluster scope: can only read namespaced object of this type
ERRO[0000] TLS certificates not provided, not setting up pod http server  certPath= keyPath= node=virtual-kubelet operatingSystem=Linux provider=openstack watchedNamespace=
Error: could not setup listener for pod metrics http server: listen tcp :10255: bind: address already in use

解决这个问题有两个方法：
1):修改local-up-cluster.sh，禁用read-only port
all_kubelet_flags 增加

"--read-only-port=0"

2):修改virtual-kubelet代码，支持禁用read-only port