说明
- 此篇的k8s基础环境搭建方法基于前几篇文章。
- MinIO Kubernetes Plugin提供了使用kubectl MinIO命令在Kubernetes集群上部署和管理MinIO租户的本地支持。
- MinIO Operator 自动生成TLS证书签名请求(Certificate Signing Requests, CSR)，并使用Kubernetes
certificates.k8s.ioTLS证书管理API，用于创建已签名的TLS证书。
一、kube-controller-manager 支持创建证书
修改 /etc/kubernetes/controller-manager 添加下面两行：
--cluster-signing-cert-file=/etc/kubernetes/pki/ca.crt \
--cluster-signing-key-file=/etc/kubernetes/pki/ca.key \
修改后的文件内容如下：
重启 kube-controller-manager 服务
systemctl restart kube-controller-manager
二、安装 minio operator 插件
- 下载最新版的 minio operator 可执行文件：
wget https://github.com/minio/operator/releases/download/v4.5.4/kubectl-minio_4.5.4_linux_amd64
- 重命名
[root@k8s0 workspace_minio]# mv kubectl-minio_4.5.4_linux_amd64 kubectl-minio
- 添加执行权限
[root@k8s0 workspace_minio]# chmod +x kubectl-minio
- 移动到
/usr/local/bin/
[root@k8s0 workspace_minio]# cp kubectl-minio /usr/local/bin/
- 验证：
[root@k8s0 workspace_minio]# kubectl minio version
v4.5.4
- 可以看到，minio operator 已经安装成功。
三、初始化 minio operator
- 执行
kubectl minio init命令初始化
[root@k8s0 workspace_minio]# kubectl minio init
namespace/minio-operator created
serviceaccount/minio-operator created
clusterrole.rbac.authorization.k8s.io/minio-operator-role created
clusterrolebinding.rbac.authorization.k8s.io/minio-operator-binding created
customresourcedefinition.apiextensions.k8s.io/tenants.minio.min.io created
service/operator created
deployment.apps/minio-operator created
serviceaccount/console-sa created
secret/console-sa-secret created
clusterrole.rbac.authorization.k8s.io/console-sa-role created
clusterrolebinding.rbac.authorization.k8s.io/console-sa-binding created
configmap/console-env created
service/console created
deployment.apps/console created
-----------------
To open Operator UI, start a port forward using this command:
kubectl minio proxy -n minio-operator
-----------------
- 运行 kubectl minio proxy -n minio-operator 命令临时将流量从minio操作员控制台服务转发到本地机器
[root@k8s0 workspace_minio]# kubectl minio proxy -n minio-operator
Starting port forward of the Console UI.
To connect open a browser and go to http://localhost:9090
Current JWT to login: eyJhbGciOiJSUzI1NiIsImtpZCI6Inl6cFRBM25HUTJrNUNsck1CR2FpNmVlaHlNaWVHV1lVdEkwOGtrZjBMckkifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJtaW5pby1vcGVyYXRvciIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJjb25zb2xlLXNhLXNlY3JldCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJjb25zb2xlLXNhIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiZmQ5ZTZjYjMtMDBiOS00NTA5LTgzMzctMzBlNjUyZWNlMDU2Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Om1pbmlvLW9wZXJhdG9yOmNvbnNvbGUtc2EifQ.djbnoBplYXwaV2dfUC0BqSZ1rCRFP85SxZdHy3Z1l0rzvKpcUn_B4O6jg6gt4noJh2c8712t_vsHTnlgUKu4ptif2vVFfcCzX21G6ILcZ-X0b6PzQ58rFeFG6RzB-0E1TZXEcaXWrXfXoplwywtVscdyvDNXSXCvwSquyrHs4o4oP2nsxhL0OLblLFT2RiA-6jpwvnaWUEBaO3paZzyvCiUcBoPgQd2HHHRih2MXRWG72w3lCjICa8z60ihKq3NTqbTB5HUJouRoDluI5ykYcajMZ6sXMDFEDG30Wa5q5NE_pXH9XtMOBFXlFiuJZhfq72aD3-snlaGoDQajMuZ_ng
Forwarding from 0.0.0.0:9090 -> 9090
- 这里其实就是创建了一个登录token，并且将临时暴露端口，相当于集成了
kubectl create token和kubectl port-forward两条命名。 - 复制 token，浏览器输入
机器IP:9090，并在登录界面输入 token，即可登录
登录成功
四、安装 DirectPV plugin 插件
- 通过 krew 安装 directpv
[root@k8s0 workspace_krew]# kubectl krew install directpv
Updated the local copy of plugin index.
Installing plugin: directpv
Installed plugin: directpv
\
| Use this plugin:
| kubectl directpv
| Documentation:
| https://github.com/minio/directpv
/
WARNING: You installed plugin "directpv" from the krew-index plugin repository.
These plugins are not audited for security by the Krew maintainers.
Run them at your own risk.
- 在 kubernetes 安装 directpv 插件
[root@k8s0 workspace_krew]# kubectl directpv install
I1125 10:32:13.441654 11816 ns.go:65] 'direct.csi.min.io' namespace created
I1125 10:32:13.458859 11816 rbac.go:352] 'direct.csi.min.io' rbac created
I1125 10:32:13.542620 11816 crd.go:135] crds successfully registered
I1125 10:32:13.556357 11816 csidriver.go:148] 'direct.csi.min.io' csidriver created
I1125 10:32:13.583081 11816 storageclass.go:48] 'direct.csi.min.io' storageclass created
I1125 10:32:13.595027 11816 service.go:38] 'direct.csi.min.io' service created
I1125 10:32:13.610337 11816 daemonset.go:41] 'direct.csi.min.io' daemonset created
I1125 10:32:13.632032 11816 deployment.go:289] 'direct.csi.min.io' deployment created
- 验证安装是否成功
[root@k8s0 tmp]# kubectl directpv info
NODE CAPACITY ALLOCATED VOLUMES DRIVES
• 192.168.3.100 - - - -
• 192.168.3.101 - - - -
• 192.168.3.102 - - - -
[root@k8s0 tmp]# date
Sat Nov 26 11:19:07 CST 2022
- 列出集群中可用的驱动器
[root@k8s0 tmp]# kubectl directpv drives ls
DRIVE CAPACITY ALLOCATED FILESYSTEM VOLUMES NODE ACCESS-TIER STATUS
/dev/dm-1 4.0 GiB - swap - 192.168.3.100 - Available
/dev/sdb2 1.4 TiB - - - 192.168.3.100 - Available
/dev/dm-1 4.0 GiB - swap - 192.168.3.101 - Available
/dev/sdb2 1.4 TiB - - - 192.168.3.101 - Available
/dev/dm-1 4.0 GiB - swap - 192.168.3.102 - Available
/dev/sdb2 1.4 TiB - - - 192.168.3.102 - Available
- 选择 directpv 应该管理和格式化的驱动器
kubectl directpv drives format --drives /dev/sdb2
- 该命令标识，格式化所有节点上的
/dev/sdb2
- 验证
[root@k8s0 tmp]# kubectl directpv drives ls
DRIVE CAPACITY ALLOCATED FILESYSTEM VOLUMES NODE ACCESS-TIER STATUS
/dev/dm-1 4.0 GiB - swap - 192.168.3.100 - Available
/dev/sdb2 1.4 TiB - xfs - 192.168.3.100 - Ready
/dev/dm-1 4.0 GiB - swap - 192.168.3.101 - Available
/dev/sdb2 1.4 TiB - xfs - 192.168.3.101 - Ready
/dev/dm-1 4.0 GiB - swap - 192.168.3.102 - Available
/dev/sdb2 1.4 TiB - xfs - 192.168.3.102 - Ready
- 可以看到所有节点上的
/dev/sdb2已经格式化完毕了。
- OK，现在可以 directpv现在可以被指定为
PodSpec.VolumeClaimTemplates中的存储类
五、创建高可用 minio 集群
现在可以通过第三步里的“Create”按钮创建集群了。