K8S NodePort无法访问MySQL

K8s单机部署填坑

1. 设置Docker存储路径

• 1.13.1 或者其它版本开始，要设置daemon.json的字段为“graph”，较早版本“data-root”

{
"registry-mirrors":["http://hub-mirror.c.163.com"],
"graph":"/data/docker"
}

2. kubectl create -f yaml后一直ContainerCreating

1. 检查后发现如下错误

   Error syncing pod 80f69868-53a8-11ea-aa0f-080027d3bf90, skipping: failed to “StartContainer” for “POD” with ErrImagePull: “image pull failed for registry.access.redhat.com/rhel7/pod-infrastructure:latest, this may be because there are no credentials on this request. details: (open /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt: no such file or directory)”

2. 诊断，

   CentOS7的pytho-rhsm-certificates被替换成空包，需要自己解决证书的问题

3. 解决

   $ sudo -i && cd /tmp $ wget http://mirror.centos.org/centos/7/os/x86_64/Packages/python-rhsm-certificates-1.19.10-1.el7_4.x86_64.rpm
   [root@vm-001]$ rpm2cpio python-rhsm-certificates-1.19.10-1.el7_4.x86_64.rpm | cpio -iv --to-stdout ./etc/rhsm/ca/redhat-uep.pem | tee /etc/rhsm/ca/redhat-uep.pem
   

3. NodePort无法访问Mysql

3.1 起因测试一个MySQL。纯粹测试，不要纠结

编辑一个mysql-rc.yaml

apiVersion: v1
kind: ReplicationController
metadata:
  name: mysql-rc
spec:
  replicas: 1
  selector:
    app: mysql-pod # 这里
  template:
    metadata:
      labels:
        app: mysql-pod
    spec:
      containers:
      - name: mysql
        image: mysql:5.6
        ports:
        - containerPort: 3306
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: "123"

编辑一个mysql-svc.yaml

apiVersion: v1
kind: Service
metadata:
  name: mysql-svc
  labels:
    name: mysql-svc
spec:
  type: NodePort
  ports:
  - port: 3306
    protocol: TCP
    targetPort: 3306
    name: http
    nodePort: 30006
  selector:
    name: mysql-pod # 注意此处

3.2 现象

1. Pod与Service都完美启动，IP，port都在

$kubectl get pod,service -o wide
NAME                READY     STATUS    RESTARTS   AGE       IP           NODE
po/mysql-rc-cr4f1   1/1       Running   0          3h        172.17.0.2   127.0.0.1

NAME             CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE       SELECTOR
svc/kubernetes   10.254.0.1       <none>        443/TCP          1d        <none>
svc/mysql-svc    10.254.121.120   <nodes>       3306:30006/TCP   3h        app=mysql-pod

2. 在节点上通过30006访问mysql没有返回，一直等待连接

[root@vm-001] $mysql -h192.168.56.4 -P30006 -uroot -p123

4. 测试Pod，可以正常访问

$kubectl describe pod -l app
Name:		mysql-rc-cr4f1
Namespace:	default
Node:		127.0.0.1/127.0.0.1
Start Time:	Fri, 21 Feb 2020 14:16:58 +0800
Labels:		app=mysql-pod
Status:		Running
IP:		172.17.0.2
Controllers:	ReplicationController/mysql-rc
Containers:
  mysql:
    Container ID:	docker://6ca5fb121250d72700da83c8365c08710fcd61b9d447ff222af9238ea967dc28 
    Image:		mysql:5.6
    Image ID:		docker-pullable://docker.io/mysql@sha256:bef096aee20d73cbfd87b02856321040ab1127e94b707b41927804776dca02fc
    Port:		3306/TCP
    State:		Running
      Started:		Fri, 21 Feb 2020 14:16:59 +0800
    Ready:		True
    Restart Count:	0
    Volume Mounts:	<none>
    Environment Variables:
      MYSQL_ROOT_PASSWORD:	123
Conditions:
  Type		Status
  Initialized 	True
  Ready 	True
  PodScheduled 	True
No volumes.
QoS Class:	BestEffort
Tolerations:	<none>
No events.

$docker exec -it 6ca5fb121250 /bin/bash
$mysql
$>

3.3 解决过程

1. 最早怀疑代理失败，通过查看kube-proxy日志，发现kube-proxy只是一个Iptables的规则同步，没有直接代理。

2. 怀疑Docker启动有问题，登入检查后发现正常

3. 查看Service调试，找到Endpoint部分

   发现Service的Endpoint为none，基本确认，Service没有找到对应的Pod

   $kubectl get endpoints mysql-svc
   NAME        ENDPOINTS   AGE
   mysql-svc   <none>      3h
   

4. 仔细查看yaml，发现Service中的selector用的是“name: mysql-pod”，而mysql-rc.yaml中用的名字是"app:mysql-pod"。

5. 修改mysql-svc.yaml的selector “app:mysql-pod”，问题解决！

3.4 总结

1. 没有注意Selector的命名

2. Service、Pod(ReplicationSet)的关系。
   Service本质是代理，Pod本质是Endpoint。通过Service的访问，通过Selector发现并接入Pod作为Service的EndPoint

3. Pod(Replication)本质是资源描述，建立Docker容器的过程

4. Service，是出口，建立代理。并选择用哪个Pod来作为Endpoint