SpringBoot集成kubernetes-client升级k8s后初始化失败问题

SpringBoot集成kubernetes-client升级k8s后初始化失败问题

1.问题描述

程序以前使用的k8s版本是1.16,fabric8.kubernetes-client的版本是4.10.2,springboot版本是2.3.5。由于环境切换,这次需要升级k8s的版本,现在将k8s版本升级到了1.26,程序中使用了新配置的certificate-authority-data,client-certificate-data等。程序中原本初始化写法如下:

config = new ConfigBuilder().withMasterUrl(server)
        .withCaCertData(certificateAuthorityData)
        .withClientCertData(clientCertificateData)
        .withClientKeyData(clientKeyData)
        .build();
LogUtil.info(LogEnum.BIZ_K8S, "config信息为{}", JSON.toJSONString(config));
client = new DefaultKubernetesClient(config);

但是程序启动缺出现了问题,问题如下:

Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.dubhe.k8s.utils.K8sUtils]: Factory method 'k8sUtils' threw exception; nested exception is io.fabric8.kubernetes.client.KubernetesClientException: An error has occurred.
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185) ~[spring-beans-5.2.10.RELEASE.jar:5.2.10.RELEASE]
	at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:650) ~[spring-beans-5.2.10.RELEASE.jar:5.2.10.RELEASE]
	... 61 common frames omitted
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: An error has occurred.
	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64) ~[kubernetes-client-4.10.2.jar:na]
	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:53) ~[kubernetes-client-4.10.2.jar:na]
	at io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:234) ~[kubernetes-client-4.10.2.jar:na]
	at io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:63) ~[kubernetes-client-4.10.2.jar:na]
	at io.fabric8.kubernetes.client.BaseClient.<init>(BaseClient.java:51) ~[kubernetes-client-4.10.2.jar:na]
	at io.fabric8.kubernetes.client.DefaultKubernetesClient.<init>(DefaultKubernetesClient.java:133) ~[kubernetes-client-4.10.2.jar:na]
	at org.dubhe.k8s.utils.K8sUtils.<init>(K8sUtils.java:93) ~[classes/:na]
	at org.dubhe.k8s.config.K8sConfig.k8sUtils(K8sConfig.java:91) ~[classes/:na]
	at org.dubhe.k8s.config.K8sConfig$$EnhancerBySpringCGLIB$$77270501.CGLIB$k8sUtils$10(<generated>) ~[classes/:na]
	at org.dubhe.k8s.config.K8sConfig$$EnhancerBySpringCGLIB$$77270501$$FastClassBySpringCGLIB$$1f42aed6.invoke(<generated>) ~[classes/:na]
	at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244) ~[spring-core-5.2.10.RELEASE.jar:5.2.10.RELEASE]
	at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331) ~[spring-context-5.2.10.RELEASE.jar:5.2.10.RELEASE]
	at org.dubhe.k8s.config.K8sConfig$$EnhancerBySpringCGLIB$$77270501.k8sUtils(<generated>) ~[classes/:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_321]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_321]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_321]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_321]
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.2.10.RELEASE.jar:5.2.10.RELEASE]
	... 62 common frames omitted
Caused by: java.io.IOException: Invalid DER: object is not integer
	at io.fabric8.kubernetes.client.internal.PKCS1Util$Asn1Object.getInteger(PKCS1Util.java:125) ~[kubernetes-client-4.10.2.jar:na]
	at io.fabric8.kubernetes.client.internal.PKCS1Util.next(PKCS1Util.java:55) ~[kubernetes-client-4.10.2.jar:na]
	at io.fabric8.kubernetes.client.internal.PKCS1Util.decodePKCS1(PKCS1Util.java:46) ~[kubernetes-client-4.10.2.jar:na]
	at io.fabric8.kubernetes.client.internal.CertUtils.handleOtherKeys(CertUtils.java:175) ~[kubernetes-client-4.10.2.jar:na]
	at io.fabric8.kubernetes.client.internal.CertUtils.loadKey(CertUtils.java:135) ~[kubernetes-client-4.10.2.jar:na]
	at io.fabric8.kubernetes.client.internal.CertUtils.createKeyStore(CertUtils.java:113) ~[kubernetes-client-4.10.2.jar:na]
	at io.fabric8.kubernetes.client.internal.CertUtils.createKeyStore(CertUtils.java:245) ~[kubernetes-client-4.10.2.jar:na]
	at io.fabric8.kubernetes.client.internal.SSLUtils.keyManagers(SSLUtils.java:128) ~[kubernetes-client-4.10.2.jar:na]
	at io.fabric8.kubernetes.client.internal.SSLUtils.keyManagers(SSLUtils.java:122) ~[kubernetes-client-4.10.2.jar:na]
	at io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:120) ~[kubernetes-client-4.10.2.jar:na]
	... 77 common frames omitted

2.问题分析

然后就上网去查找资料,但是搜索了很多,都没有找到有用的信息。只好自己一点点debug去查看源代码。debug的时候看到build的时候会先去读配置文件,这让我想到了之前做的一个项目,那个项目使用的是io.kubernetes client-java这个依赖包。我再开发的时候会将config文件放在系统路径下。又换了一种创建方式:

Config config = new ConfigBuilder().build();

于是按照这种方式也将config放在相应的路径下,程序果然可以去建立连接,自测创建pod也是成功的。

由于一个读取的是配置文件,一个读取的是配置在nacos配置中心的配置,两个的配置信息毫无差别,但是执行结果就是不一样。然后再去debug去看两中配置问题到底有何不一样。最后发现Config中的clientKeyAlgo中赋值不同,读取nacos中的配置产生的算法是RSA,读取config配置文件产生的是EC。

然后我将系统路径下的config文件移除,将原本程序改为:

Config config = new ConfigBuilder().withMasterUrl(server)         .withCaCertData(certificateAuthorityData)         .withClientCertData(clientCertificateData)         .withClientKeyData(clientKeyData)         .withClientKeyAlgo("EC")         .build();

发现也成功了。由于时间关系,就没有仔细去看为什么读取nacos中的配置产生的是RSA,读取config配置文件产生的是EC。

3.解决办法

最终解决方案是:

1.在程序启动的包设置config环境变量,让程序读取config文件;设置了config文件后,就可以用Config config = new ConfigBuilder().build();这种方式创建。

2.依然去读取nacos中的配置(certificate-authority-data,client-certificate-data),在程序Builder时手动设置ClientKeyAlgo。

你可能感兴趣的:(SpringBoot,k8s,spring,boot,kubernetes,后端)