Kubernetes安装系列之tls方式下的kubectl设定

这篇文章整理一下apiserver的缺省8080端口关闭的方法，以及这种方式下kubectl的设定方式，本文以脚本的方式进行固化，内容仍然放在github的easypack上。

整体操作

• https://blog.csdn.net/liumiaocn/article/details/88413428

相关参数

apiserver的insecure-port，缺省值为8080，在后续版本中将会移除，建议先行关闭。

      --insecure-port int          The port on which to serve unsecured, unauthenticated access. (default 8080) (DEPRECATED: This flag will be removed in a future version.)

insecure方式下的监听进程

使用kubectl cluster-info可以确认到使用的是8080端口

[root@host131 shell]# kubectl cluster-info
Kubernetes master is running at http://localhost:8080

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
[root@host131 shell]#

确认进程状况也会发现kube-apiserver同时在6443（TLS）和8080提供服务。

[root@host131 shell]# netstat -tunlp |grep kube-apiserver
tcp        0      0 192.168.163.131:6443    0.0.0.0:*               LISTEN      1210/kube-apiserver 
tcp        0      0 127.0.0.1:8080          0.0.0.0:*               LISTEN      1210/kube-apiserver 
[root@host131 shell]# 

关闭8080端口的监听

可以通过--insecure-port=0将缺省的8080端口进行关闭。
重新设定并启动apiserver，确认进程状况也会发现kube-apiserver此时只在6443（TLS）提供服务。

[root@host131 shell]# netstat -tunlp |grep kube-apiserver
tcp        0      0 192.168.163.131:6443    0.0.0.0:*               LISTEN      2739/kube-apiserver 
[root@host131 shell]#

执行kubectl也会提示如下错误信息

[root@host131 shell]# kubectl version
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.4", GitCommit:"c27b913fddd1a6c480c229191a087698aa92f0b1", GitTreeState:"clean", BuildDate:"2019-02-28T13:37:52Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
The connection to the server localhost:8080 was refused - did you specify the right host or port?
[root@host131 shell]# 
[root@host131 shell]# kubectl cluster-info

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
The connection to the server localhost:8080 was refused - did you specify the right host or port?
[root@host131 shell]#

csr文件

使用如下csr文件创建证书

[root@host131 shell]# cat /etc/ssl/ca/admin-csr.json 
{
    "CN": "admin",
    "hosts": [],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "DaLian",
            "ST": "LiaoNing",
            "O": "system:masters",
            "OU": "System"
        }
    ]
}
[root@host131 shell]#

脚本示例

生成证书和kubeconfig设定的脚本如下：

[root@host131 shell]# cat step1-2-prepare-admin-cert.sh 
#!/bin/sh

. ./install.cfg

# set cfssl tools in search path
chmod 755 ${ENV_HOME_CFSSL}/*
if [ $? -ne 0 ]; then
  echo "prepare downloaded cfssl tools in ${ENV_HOME_CFSSL} in advance"
  exit
fi

export PATH=${ENV_HOME_CFSSL}:$PATH

# create dir for certs when not existing
mkdir -p ${ENV_SSL_CA_DIR} ${ENV_SSL_ETCD_DIR}

# create csr files of admin
cat <<EOF >${ENV_SSL_CA_DIR}/${ENV_SSL_FILE_ADMIN_CSR}
{
    "CN": "${ENV_SSL_ADMIN_CN}",
    "hosts": [],
    "key": {
        "algo": "${ENV_SSL_KEY_ALGO}",
        "size": ${ENV_SSL_KEY_SIZE}
    },
    "names": [
        {
            "C": "${ENV_SSL_NAMES_C}",
            "L": "${ENV_SSL_NAMES_L}",
            "ST": "${ENV_SSL_NAMES_ST}",
            "O": "${ENV_SSL_NAMES_O_MASTER}",
            "OU": "${ENV_SSL_NAMES_OU}"
        }
    ]
}
EOF

ODIR=`pwd`
cd ${ENV_SSL_CA_DIR}

cfssl gencert -ca=${ENV_SSL_CA_DIR}/${ENV_SSL_FILE_CA_PEM} -ca-key=${ENV_SSL_CA_DIR}/${ENV_SSL_FILE_CA_KEY} -config=${ENV_SSL_CA_DIR}/${ENV_SSL_FILE_CA_CONFIG} -profile=${ENV_SSL_PROFILE_K8S} ${ENV_SSL_FILE_ADMIN_CSR} | cfssljson -bare ${ENV_SSL_ADMIN_CERT_PRIFIX}

# confirm cert pem: kubeadmin-key.pem  kubeadmin.pem
ls ${ENV_SSL_CA_DIR}/${ENV_SSL_ADMIN_CERT_PRIFIX}*.pem

#echo "openssl pkcs12 -export -out ${ENV_SSL_ADMIN_CERT_PRIFIX}.pfx -inkey ${ENV_SSL_ADMIN_CERT_PRIFIX}-key.pem -in ${ENV_SSL_ADMIN_CERT_PRIFIX}.pem -certfile ${ENV_SSL_FILE_CA_PEM}"

# Create kubectl kubeconfig file. 
kubectl config set-cluster ${ENV_KUBECONFIG_CLUSTER} \
  --certificate-authority=${ENV_SSL_CA_DIR}/${ENV_SSL_FILE_CA_PEM} \
  --embed-certs=${ENV_KUBECONFIG_EMBED_CERTS} \
  --server=${ENV_KUBE_MASTER_HTTPS} \
  --kubeconfig=${ENV_KUBECONFIG_KUBECTL}

kubectl config set-credentials ${ENV_KUBECONFIG_CLIENT_KUBECTL} \
  --client-certificate=${ENV_SSL_CA_DIR}/${ENV_SSL_ADMIN_CERT_PRIFIX}.pem \
  --client-key=${ENV_SSL_CA_DIR}/${ENV_SSL_ADMIN_CERT_PRIFIX}-key.pem \
  --embed-certs=${ENV_KUBECONFIG_EMBED_CERTS} \
  --kubeconfig=${ENV_KUBECONFIG_KUBECTL}

kubectl config set-context ${ENV_KUBECONFIG_CLUSTER} \
  --cluster=${ENV_KUBECONFIG_CLUSTER} \
  --user=${ENV_KUBECONFIG_CLIENT_KUBECTL} \
  --kubeconfig=${ENV_KUBECONFIG_KUBECTL}

kubectl config use-context ${ENV_KUBECONFIG_CLUSTER} --kubeconfig=${ENV_KUBECONFIG_KUBECTL}

echo "## copy ${ENV_KUBECONFIG_KUBECTL} to ~/.kube/config"
cp ${ENV_SSL_CA_DIR}/${ENV_KUBECONFIG_KUBECTL} ~/.kube/config
[root@host131 shell]#

执行示例

[root@host131 shell]# sh step1-2-prepare-admin-cert.sh 
2019/03/29 20:34:54 [INFO] generate received request
2019/03/29 20:34:54 [INFO] received CSR
2019/03/29 20:34:54 [INFO] generating key: rsa-2048
2019/03/29 20:34:55 [INFO] encoded CSR
2019/03/29 20:34:55 [INFO] signed certificate with serial number 29307085123257831389398834028051059242701418302
2019/03/29 20:34:55 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
/etc/ssl/ca/kubeadmin-key.pem  /etc/ssl/ca/kubeadmin.pem
Cluster "kubernetes" set.
User "admin" set.
Context "kubernetes" modified.
Switched to context "kubernetes".
## copy kubectl.kubeconfig to ~/.kube/config
[root@host131 shell]# 

结果确认

执行之后，kubectl可以正常动作了，因为kubeconfig在生成的时候，选择了将证书嵌入其中，如果master节点有多个的情况，这是只需要将此文件拷贝到其他节点的~/.kube/config中即可使用。

[root@host131 shell]# kubectl version
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.4", GitCommit:"c27b913fddd1a6c480c229191a087698aa92f0b1", GitTreeState:"clean", BuildDate:"2019-02-28T13:37:52Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.4", GitCommit:"c27b913fddd1a6c480c229191a087698aa92f0b1", GitTreeState:"clean", BuildDate:"2019-02-28T13:30:26Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
[root@host131 shell]# 
[root@host131 shell]# kubectl cluster-info
Kubernetes master is running at https://192.168.163.131:6443

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
[root@host131 shell]#