kubernetes学习:3.创建tls证书和秘钥
kubernete tls证书和秘钥
传输k8s各组件之间的通信可以使用http方式，但是为了安全起见，生产环境需要使用https方式通信，所以我们需要生成tls证书进行加密传输。
构建环境
再次介绍下我们本次搭建的环境：
| 节点名称 | ip | 配置 |
|---|---|---|
| wecloud-test-k8s-1（master） | 192.168.99.183 | 4核，4G，50G磁盘 |
| wecloud-test-k8s-2（node1） | 192.168.99.189 | 4核，4G，50G磁盘 |
| wecloud-test-k8s-3（node2） | 192.168.99.185 | 4核，4G，50G磁盘 |
| wecloud-test-k8s-4（node3） | 192.168.99.196 | 4核，4G，50G磁盘 |
生成tls证书的工具:cfssl
生成的 CA 证书和秘钥文件如下：
ca-key.pem
ca.pem
kubernetes-key.pem
kubernetes.pem
kube-proxy.pem
kube-proxy-key.pem
admin.pem
admin-key.pem
使用证书的组件如下：
etcd：使用 ca.pem、kubernetes-key.pem、kubernetes.pem；
kube-apiserver：使用 ca.pem、kubernetes-key.pem、kubernetes.pem；
kubelet：使用 ca.pem；
kube-proxy：使用 ca.pem、kube-proxy-key.pem、kube-proxy.pem；
kubectl：使用 ca.pem、admin-key.pem、admin.pem；
kube-controller-manager：使用 ca-key.pem、ca.pem
证书的创建过程是在192.168.99.183（master）节点上进行的，生成好证书只需要把证书分发给其他节点，这样就可以进行加密通信了。
安装tls和秘钥
安装cfssl
直接在官网下载cfssl相关二进制包：
wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64增加可执行权限后，将可执行二进制文件拷贝到全局变量可以识别的目录/usr/local/bin:
[root@wecloud-test-k8s-1 ~]# chmod +x cfssl-certinfo_linux-amd64
[root@wecloud-test-k8s-1 ~]# chmod +x cfssljson_linux-amd64
[root@wecloud-test-k8s-1 ~]# chmod +x cfssl_linux-amd64
[root@wecloud-test-k8s-1 ~]# mv cfssl_linux-amd64 /usr/local/bin/cfssl
[root@wecloud-test-k8s-1 ~]# mv cfssljson_linux-amd64 /usr/local/bin/cfssljson
[root@wecloud-test-k8s-1 ~]# mv cfssl-certinfo_linux-amd64 /usr/local/bin/cfssl-certinfo创建CA文件
创建 CA 配置文件
[root@wecloud-test-k8s-1 ~]# mkdir /root/ssl
[root@wecloud-test-k8s-1 ~]# cd /root/ssl/
[root@wecloud-test-k8s-1 ssl]# cfssl print-defaults config > config.json
[root@wecloud-test-k8s-1 ssl]# cfssl print-defaults csr > csr.json生成配置文件模板
cat > ca-config.json <<EOF
{
"signing": {
"default": {
"expiry": "87600h"
},
"profiles": {
"kubernetes": {
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
],
"expiry": "87600h"
}
}
}
}
EOFexpiry： 其中expiry是过期时间，可以指定多个;
signing： 表示该证书可以签名其他证书;
server auth： 表示客户端可以用该证书对服务端进行验证;
client auth： 表示服务端可以用该证书对客户端进行验证;
创建CA证书签名请求
创建ca-csr.json文件：
{
"CN": "kubernetes",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "ZheJiang",
"L": "HangZhou",
"O": "k8s",
"OU": "System"
}
]
}- “CN”：Common Name，kube-apiserver 从证书中提取该字段作为请求的用户名 (User Name)；浏览器使用该字段验证网站是否合法；
- “O”：Organization，kube-apiserver 从证书中提取该字段作为请求用户所属的组 (Group)；
生成 CA 证书和私钥
[root@wecloud-test-k8s-1 ssl]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca
2018/04/01 17:16:31 [INFO] generating a new CA key and certificate from CSR
2018/04/01 17:16:31 [INFO] generate received request
2018/04/01 17:16:31 [INFO] received CSR
2018/04/01 17:16:31 [INFO] generating key: rsa-2048
2018/04/01 17:16:31 [INFO] encoded CSR
2018/04/01 17:16:31 [INFO] signed certificate with serial number 249255656321361266442489392967889104825701924957
[root@wecloud-test-k8s-1 ssl]# ls
ca-config.json ca.csr ca-csr.json ca-key.pem ca.pem config.json csr.json ca-key.pem 和 ca.pem分别为生成的私钥和证书。
创建kubernetes证书
创建 kubernetes 证书签名请求文件 kubernetes-csr.json：
{
"CN": "kubernetes",
"hosts": [
"127.0.0.1",
"192.168.99.183",
"192.168.99.189",
"192.168.99.185",
"192.168.99.196",
"10.254.0.1",
"kubernetes",
"kubernetes.default",
"kubernetes.default.svc",
"kubernetes.default.svc.cluster",
"kubernetes.default.svc.cluster.local"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "ZheJiang",
"L": "HangZhou",
"O": "k8s",
"OU": "System"
}
]
}hosts 字段不为空则需要指定授权使用该证书的 IP 或域名列表,把我们整个集群需要的节点都添加进去。
生成kubernetes证书和私钥
[root@wecloud-test-k8s-1 ssl]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kubernetes-csr.json | cfssljson -bare kubernetes查看生成结果：
[root@wecloud-test-k8s-1 ssl]# ls
ca-config.json ca-csr.json ca.pem csr.json kubernetes-csr.json kubernetes.pem
ca.csr ca-key.pem config.json kubernetes.csr kubernetes-key.pem创建 admin 证书
创建 admin 证书签名请求文件 admin-csr.json：
{
"CN": "admin",
"hosts": [],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "BeiJing",
"L": "BeiJing",
"O": "system:masters",
"OU": "System"
}
]
}后续 kube-apiserver 使用 RBAC 对客户端(如 kubelet、kube-proxy、Pod)请求进行授权；
kube-apiserver 预定义了一些 RBAC 使用的 RoleBindings，如 cluster-admin 将 Group system:masters 与 Role cluster-admin 绑定，该 Role 授予了调用kube-apiserver 的所有 API的权限；
O 指定该证书的 Group 为 system:masters，kubelet 使用该证书访问 kube-apiserver 时 ，由于证书被 CA 签名，所以认证通过，同时由于证书用户组为经过预授权的 system:masters，所以被授予访问所有 API 的权限；
生成 admin 证书和私钥
[root@wecloud-test-k8s-1 ssl]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin
2018/04/01 17:52:44 [INFO] generate received request
2018/04/01 17:52:44 [INFO] received CSR
2018/04/01 17:52:44 [INFO] generating key: rsa-2048
2018/04/01 17:52:45 [INFO] encoded CSR
2018/04/01 17:52:45 [INFO] signed certificate with serial number 304064711980818042733176492155385650115390405387
2018/04/01 17:52:45 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
[root@wecloud-test-k8s-1 ssl]# ls
admin.csr admin-key.pem ca-config.json ca-csr.json ca.pem csr.json kubernetes-csr.json kubernetes.pem
admin-csr.json admin.pem ca.csr ca-key.pem config.json kubernetes.csr kubernetes-key.pem
创建 kube-proxy 证书
创建 kube-proxy 证书签名请求文件 kube-proxy-csr.json：
{
"CN": "system:kube-proxy",
"hosts": [],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "BeiJing",
"L": "BeiJing",
"O": "k8s",
"OU": "System"
}
]
}CN 指定该证书的 User 为 system:kube-proxy；
kube-apiserver 预定义的 RoleBinding cluster-admin 将User system:kube-proxy 与 Role system:node-proxier 绑定，该 Role 授予了调用 kube-apiserver Proxy 相关 API 的权限；
[root@wecloud-test-k8s-1 ssl]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy
2018/04/01 18:04:12 [INFO] generate received request
2018/04/01 18:04:12 [INFO] received CSR
2018/04/01 18:04:12 [INFO] generating key: rsa-2048
2018/04/01 18:04:13 [INFO] encoded CSR
2018/04/01 18:04:13 [INFO] signed certificate with serial number 110046110319381529194553706381534768799655332669
2018/04/01 18:04:13 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
[root@wecloud-test-k8s-1 ssl]# ls
admin.csr admin.pem ca-csr.json config.json kube-proxy-csr.json kubernetes.csr kubernetes.pem
admin-csr.json ca-config.json ca-key.pem csr.json kube-proxy-key.pem kubernetes-csr.json
admin-key.pem ca.csr ca.pem kube-proxy.csr kube-proxy.pem kubernetes-key.pem校验证书
以 kubernetes 证书为例
使用 opsnssl 命令
[root@wecloud-test-k8s-1 ssl]# openssl x509 -noout -text -in kubernetes.pem
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
53:0b:be:1f:9d:f1:87:2c:32:29:e7:dd:ec:c7:7b:62:ad:21:24:3b
Signature Algorithm: sha256WithRSAEncryption
Issuer: C=CN, ST=ZheJiang, L=HangZhou, O=k8s, OU=System, CN=kubernetes
Validity
Not Before: Apr 1 09:25:00 2018 GMT
Not After : Mar 29 09:25:00 2028 GMT
Subject: C=CN, ST=ZheJiang, L=HangZhou, O=k8s, OU=System, CN=kubernetes
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:ac:8f:43:bf:e3:9e:a8:1a:4e:43:85:82:1c:ab:
d3:33:8a:f3:7f:13:fe:0c:13:ce:6e:95:9b:fb:be:
4a:ce:52:75:b0:5d:11:27:3a:b5:6e:83:cb:dd:f9:
0b:ee:ee:a9:3b:78:ce:af:03:68:24:8c:76:b3:44:
a2:1e:71:6f:3c:3b:97:ff:29:7a:95:7c:25:3f:a2:
91:83:8e:75:2f:d1:7e:02:39:23:e8:f4:47:38:d2:
6c:9d:d2:80:39:af:d5:b6:cb:cf:da:9b:4f:0e:8e:
1f:9e:c0:72:68:47:58:c7:e4:95:3c:74:02:ae:5b:
ea:e4:1e:f1:1c:9d:0c:8f:f6:ca:4d:2f:33:bc:31:
9a:98:73:0c:b8:c9:9c:3f:5e:5b:94:0c:af:2c:ee:
d0:9e:98:e7:dd:d7:af:a4:b0:2e:23:6f:69:2f:8d:
fc:06:41:2f:fc:26:37:18:77:16:89:cc:4e:a8:ba:
bb:bd:06:4e:48:57:ff:78:3f:f5:2d:e7:05:67:5f:
50:d8:ba:ab:40:d5:7b:82:0f:5b:57:bd:00:8d:83:
11:da:da:fe:d1:3f:74:f2:74:fc:59:60:e7:e4:47:
41:c1:c2:61:c4:35:d2:f3:f4:ba:d9:00:70:a6:37:
9c:eb:28:96:e1:34:4c:11:66:9b:9b:6e:86:68:32:
6a:c7
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Key Usage: critical
Digital Signature, Key Encipherment
X509v3 Extended Key Usage:
TLS Web Server Authentication, TLS Web Client Authentication
X509v3 Basic Constraints: critical
CA:FALSE
X509v3 Subject Key Identifier:
A3:58:12:F2:46:F6:72:15:5D:35:C6:5E:05:5B:23:D1:B6:C4:A2:E1
X509v3 Authority Key Identifier:
keyid:B9:3B:C0:2D:B4:54:68:AD:69:72:59:51:46:41:54:6B:14:22:FA:33
X509v3 Subject Alternative Name:
DNS:kubernetes, DNS:kubernetes.default, DNS:kubernetes.default.svc, DNS:kubernetes.default.svc.cluster, DNS:kubernetes.default.svc.cluster.local, IP Address:127.0.0.1, IP Address:192.168.99.183, IP Address:192.168.99.189, IP Address:192.168.99.185, IP Address:192.168.99.196, IP Address:10.254.0.1
Signature Algorithm: sha256WithRSAEncryption
63:15:06:34:bc:d0:f4:d3:65:f2:01:95:71:8d:d4:d1:6a:6b:
cc:92:f6:27:dc:34:2d:11:4b:01:78:54:b9:71:df:40:1e:32:
5d:98:0a:4f:69:a4:0a:8c:42:3a:de:d2:05:6f:90:c6:ce:1b:
fb:69:28:71:4c:0d:84:64:8e:58:9c:ec:60:fb:52:e7:cb:97:
dd:45:c6:b3:1a:f2:d0:1b:d6:78:5d:24:10:02:df:ae:72:de:
6c:c4:cb:e8:8a:f5:50:5c:5c:66:0d:79:4f:95:da:12:5a:ef:
bf:6c:86:0a:29:ce:c1:f8:62:18:95:2f:03:3d:7d:84:81:5d:
59:47:1e:6a:78:03:d4:cb:4e:dc:a0:c4:29:54:23:43:a0:1a:
ab:17:bf:9e:fe:2b:a0:0d:53:49:60:f4:d9:a6:01:b3:83:77:
17:f8:5d:9a:c7:26:0a:fe:50:51:19:b1:b6:69:eb:d3:7a:aa:
db:26:a3:fe:2a:b9:89:fb:d1:c5:d5:ad:f2:05:33:83:64:9f:
fe:5d:fc:2e:34:77:92:4b:2b:fe:86:65:7c:65:14:09:51:f6:
de:36:b4:93:34:f9:27:60:05:17:c5:c9:4d:e3:65:f1:69:85:
ec:ef:f9:33:69:70:4b:b8:a0:62:c7:dd:75:89:d6:e6:6d:00:
4a:7c:1d:8e
- 确认 Issuer 字段的内容和 ca-csr.json 一致；
- 确认 Subject 字段的内容和 kubernetes-csr.json 一致；
- 确认 X509v3 Subject Alternative Name 字段的内容和 kubernetes-csr.json 一致；
- 确认 X509v3 Key Usage、Extended Key Usage 字段的内容和 ca-config.json 中 kubernetes profile 一致；
分发证书
将生成的证书和密钥文件拷贝到所有机器的/etc/kubernetes/ssl目录下：
[root@wecloud-test-k8s-1 ssl]# mkdir /etc/kubernetes/ssl
[root@wecloud-test-k8s-1 ssl]# cp *.pem /etc/kubernetes/ssl/拷贝到另外的几个节点（在此之前需要实现master到其他节点的免密登录）：
[root@wecloud-test-k8s-1 ssl]# scp *.pem 192.168.99.189:/etc/kubernetes/ssl/
[root@wecloud-test-k8s-1 ssl]# scp *.pem 192.168.99.185:/etc/kubernetes/ssl/
[root@wecloud-test-k8s-1 ssl]# scp *.pem 192.168.99.196:/etc/kubernetes/ssl/小结
在后续的章节中需要使用上述的证书完成各组件之间的加密通信。所以这个是初始必备的操作。