How to configure certs for HA cluster for v1.15.0

In a Kubernetes cluster, there are several different components, such as etcd, api-service, scheduler, controller, Kube-proxy, Kubelet etc, lots of communication will happen among all these components, its a very important to make sure all these communication secured, so setup tls certs to secure all these communication is another important task when creating a K8s cluster, by default, K8s will generate tls certs automatically with only 1 year duration, this means 1 year later, all these certs will be expired, we need to follow some process to renew these certs otherwise your K8s cluster will stop working. in this blog, I will describe how to use cfssl tool to generate tls certs and use these certs for new K8s cluster.

install cfssl:

    curl -s -L -o /bin/cfssl https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 
    curl -s -L -o /bin/cfssljson https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 
    curl -s -L -o /bin/cfssl-certinfo https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 
    chmod +x /bin/cfssl*

Write configuration file:

    CA cert configuration
    root@ppydalbik0103:/etc/kubernetes/certs# more ca-config.json
    {
        "signing": {
            "default": {
                "expiry": "87600h"
            },
            "profiles": {
                "server": {
                    "expiry": "87600h",
                    "usages": [
                        "signing",
                        "key encipherment",
                        "server auth"
                    ]
                },
                "client": {
                    "expiry": "87600h",
                    "usages": [
                        "signing",
                        "key encipherment",
                        "client auth"
                    ]
                },
                "peer": {
                    "expiry": "87600h",
                    "usages": [
                        "signing",
                        "key encipherment",
                        "server auth",
                        "client auth"
                    ]
                },
                "kubernetes": {
                    "expiry": "87600h",
                    "usages": [
                        "signing",
                        "key encipherment",
                        "server auth",
                        "client auth"
                    ]                
                }
            }
        }
    }
    
    ca cert csr configuration(certificate signing request)
    root@ppydalbik0103:/etc/kubernetes/certs# more ca-csr.json
    {
        "CN": "Kubernetes",
        "key": {
            "algo": "rsa",
            "size": 2048
        },
        "names": [
            {
              "C": "US",
              "ST": "Dallas, TX",
              "L": "Dallas, TX",
              "O": "k8s",
              "OU": "System"
            }
        ]
    }
    
    kubernetes certs csr file
    root@ppydalbik0103:/etc/kubernetes/certs# more certs-csr.json
   {
  "CN": "kubernetes",
  "hosts": [
  "127.0.0.1",
  "::1",
  "10.94.xxx.xxx",
  "10.95.xxx.xxx",
  "10.94.xxx.xxx",
  "10.94.xxx.xxx",
  "172.17.0.1",
  "ppydalbik0101.xxx.xxx.xxx.com",
  "ppydalbik0102.xxx.xxx.xxx.com",
  "ppydalbik0103.xxx.xxx.xxx.com",
  "ppydalbik0104.xxx.xxx.xxx.com",
  "ppydalbik0101",
  "ppydalbik0102",
  "ppydalbik0103",
  "ppydalbik0104",
  "localhost",
  "kubernetes",
  "kubernetes.default",
  "kubernetes.default.svc",
  "kubernetes.default.svc.cluster",
  "kubernetes.default.svc.cluster.local"
],    
"key": {
    "algo": "rsa",
    "size": 2048
},
"names": [
    {
      "C": "US",
      "ST": "Dallas, TX",
      "L": "Dallas, TX",
      "O": "k8s",
      "OU": "System"
    }
]
}

generate CA certs:

    cfssl gencert -initca ca-csr.json | cfssljson -bare ca
    cfssl gencert -initca ca-csr.json | cfssljson -bare front-proxy-ca
    cfssl gencert -initca ca-csr.json | cfssljson -bare etcd-ca
    
    
    ##generate all the srever certs:
    cfssl gencert -ca=ca.pem -ca-key=ca-key.pem --config=ca-config.json -profile=server certs-csr.json | cfssljson -bare apiserver
    cfssl gencert -ca=etcd-ca.pem -ca-key=etcd-ca-key.pem --config=ca-config.json -profile=kubernetes certs-csr.json | cfssljson -bare server
    cfssl gencert -ca=etcd-ca.pem -ca-key=etcd-ca-key.pem --config=ca-config.json -profile=peer certs-csr.json | cfssljson -bare peer
    
    ##generate all the clients certs:
    cfssl gencert -ca=ca.pem -ca-key=ca-key.pem --config=ca-config.json -profile=client certs-csr.json | cfssljson -bare apiserver-kubelet-client
    cfssl gencert -ca=front-proxy-ca.pem -ca-key=front-proxy-ca-key.pem --config=ca-config.json -profile=client certs-csr.json | cfssljson -bare front-proxy-client
    cfssl gencert -ca=etcd-ca.pem -ca-key=etcd-ca-key.pem --config=ca-config.json -profile=client certs-csr.json | cfssljson -bare apiserver-etcd-client
    cfssl gencert -ca=etcd-ca.pem -ca-key=etcd-ca-key.pem --config=ca-config.json -profile=client certs-csr.json | cfssljson -bare healthcheck-client
                    
    ##deploy new certs to kuberenetes certs folder:
    mkdir -p /etc/kubernetes/pki/etcd
    cp /etc/kubernetes/certs/front-proxy-client.pem /etc/kubernetes/pki/front-proxy-client.crt
    cp /etc/kubernetes/certs/front-proxy-client-key.pem /etc/kubernetes/pki/front-proxy-client.key
    
    cp /etc/kubernetes/certs/front-proxy-ca.pem /etc/kubernetes/pki/front-proxy-ca.crt
    cp /etc/kubernetes/certs/front-proxy-ca-key.pem /etc/kubernetes/pki/front-proxy-ca.key
    
    cp /etc/kubernetes/certs/apiserver-kubelet-client.pem /etc/kubernetes/pki/apiserver-kubelet-client.crt
    cp /etc/kubernetes/certs/apiserver-kubelet-client-key.pem /etc/kubernetes/pki/apiserver-kubelet-client.key
    
    cp /etc/kubernetes/certs/apiserver.pem /etc/kubernetes/pki/apiserver.crt
    cp /etc/kubernetes/certs/apiserver-key.pem /etc/kubernetes/pki/apiserver.key
    
    cp /etc/kubernetes/certs/apiserver-etcd-client.pem /etc/kubernetes/pki/apiserver-etcd-client.crt
    cp /etc/kubernetes/certs/apiserver-etcd-client-key.pem /etc/kubernetes/pki/apiserver-etcd-client.key
    
    cp /etc/kubernetes/certs/ca.pem /etc/kubernetes/pki/ca.crt
    cp /etc/kubernetes/certs/ca-key.pem /etc/kubernetes/pki/ca.key
    
    cp /etc/kubernetes/certs/etcd-ca.pem /etc/kubernetes/pki/etcd/ca.crt
    cp /etc/kubernetes/certs/etcd-ca-key.pem /etc/kubernetes/pki/etcd/ca.key
    
    cp /etc/kubernetes/certs/healthcheck-client.pem /etc/kubernetes/pki/etcd/healthcheck-client.crt
    cp /etc/kubernetes/certs/healthcheck-client-key.pem /etc/kubernetes/pki/etcd/healthcheck-client.key
    
    cp /etc/kubernetes/certs/server.pem /etc/kubernetes/pki/etcd/server.crt
    cp /etc/kubernetes/certs/server-key.pem /etc/kubernetes/pki/etcd/server.key
    
    cp /etc/kubernetes/certs/peer.pem /etc/kubernetes/pki/etcd/peer.crt
    cp /etc/kubernetes/certs/peer-key.pem /etc/kubernetes/pki/etcd/peer.key

copy all the certs to all master nodes:

root@ppydalbik0102:/etc/kubernetes/pki# ls -lt
total 52
drwx------ 2 root root 4096 Jul 19 02:59 etcd
-rw------- 1 root root 1679 Jul 19 02:59 ca.key
-rw------- 1 root root 1375 Jul 19 02:59 ca.crt
-rw------- 1 root root 1675 Jul 19 02:59 apiserver-etcd-client.key
-rw------- 1 root root 1838 Jul 19 02:59 apiserver-etcd-client.crt
-rw------- 1 root root 1679 Jul 19 02:59 apiserver.key
-rw------- 1 root root 1838 Jul 19 02:59 apiserver.crt
-rw------- 1 root root 1838 Jul 19 02:59 apiserver-kubelet-client.crt
-rw------- 1 root root 1675 Jul 19 02:59 apiserver-kubelet-client.key
-rw------- 1 root root 1679 Jul 19 02:59 front-proxy-ca.key
-rw------- 1 root root 1375 Jul 19 02:59 front-proxy-ca.crt
-rw------- 1 root root 1679 Jul 19 02:59 front-proxy-client.key
-rw------- 1 root root 1838 Jul 19 02:59 front-proxy-client.crt
root@ppydalbik0102:/etc/kubernetes/pki# cd etcd
root@ppydalbik0102:/etc/kubernetes/pki/etcd# ls -lt
total 32
-rw------- 1 root root 1679 Jul 19 02:59 peer.key
-rw------- 1 root root 1850 Jul 19 02:59 peer.crt
-rw------- 1 root root 1675 Jul 19 02:59 server.key
-rw------- 1 root root 1838 Jul 19 02:59 server.crt
-rw------- 1 root root 1675 Jul 19 02:59 healthcheck-client.key
-rw------- 1 root root 1838 Jul 19 02:59 healthcheck-client.crt
-rw------- 1 root root 1375 Jul 19 02:59 ca.crt
-rw------- 1 root root 1675 Jul 19 02:59 ca.key

then we can use Kubeadm to initialize cluster.

kubeadm init --config=/etc/kubernetes/k8s-cluster-bi.yaml --upload-certs

check certs status:

    kubeadm alpha certs check-expiration
    CERTIFICATE                EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
    admin.conf                 Jul 17, 2020 08:46 UTC   364d            no      
    apiserver                  Jul 15, 2029 07:53 UTC   9y              no      
    apiserver-etcd-client      Jul 15, 2029 08:38 UTC   9y              no      
    apiserver-kubelet-client   Jul 15, 2029 08:25 UTC   9y              no      
    controller-manager.conf    Jul 17, 2020 08:46 UTC   364d            no      
    etcd-healthcheck-client    Jul 15, 2029 08:41 UTC   9y              no      
    etcd-peer                  Jul 15, 2029 08:40 UTC   9y              no      
    etcd-server                Jul 15, 2029 08:39 UTC   9y              no      
    front-proxy-client         Jul 15, 2029 08:35 UTC   9y              no      
    scheduler.conf             Jul 17, 2020 08:46 UTC   364d            no   

attention there are 3 client certs with 1 year expiration in .conf files, these client certs will be refreshed automatically when they will be expired.