dubbo-admin 因连接ZooKeeper超时启动失败问题

一、遇见的问题

1.1 问题背景

dubbo-admin开源项目地址：https://github.com/apache/dubbo-admin/blob/master/README_ZH.md

尝试本地运行 dubbo-admin 项目，项目的后端部分是一个纯SpringBoot工程，按理说只需要配置一下我的注册中心的地址就可以在本地运行。但是每次运行都会报错。

1.2 问题描述

1. 启动失败，报错信息为 failed to connect to zookeeper server 下载下来的Dubbo-admin项目，运行其 dubbo-admin-server 的SpringBoot项目启动失败。下载下来后仅仅修改过三个注册相关的Zookeeper地址。

# centers in dubbo2.7, if you want to add parameters, please add them to the url
admin.registry.address=
admin.config-center=
admin.metadata-report.address=

报错信息

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2023-07-13 02:15:59.659 ERROR 2712 --- [           main] o.s.boot.SpringApplication               : Application run failed

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'getServiceDiscoveryRegistry' defined in class path resource [org/apache/dubbo/admin/config/ConfigCenter.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.apache.dubbo.registry.client.ServiceDiscovery]: Factory method 'getServiceDiscoveryRegistry' threw exception; nested exception is java.lang.IllegalStateException: Create zookeeper service discovery failed.
	at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:657)
	at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:485)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1341)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1181)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:556)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:324)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:897)
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:879)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:551)
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:143)
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:755)
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747)
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:402)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:312)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1247)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1236)
	at org.apache.dubbo.admin.DubboAdminApplication.main(DubboAdminApplication.java:35)
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.apache.dubbo.registry.client.ServiceDiscovery]: Factory method 'getServiceDiscoveryRegistry' threw exception; nested exception is java.lang.IllegalStateException: Create zookeeper service discovery failed.
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185)
	at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:652)
	... 20 common frames omitted
Caused by: java.lang.IllegalStateException: Create zookeeper service discovery failed.
	at org.apache.dubbo.registry.zookeeper.ZookeeperServiceDiscovery.<init>(ZookeeperServiceDiscovery.java:77)
	at org.apache.dubbo.registry.zookeeper.ZookeeperServiceDiscoveryFactory.createDiscovery(ZookeeperServiceDiscoveryFactory.java:27)
	at org.apache.dubbo.registry.client.AbstractServiceDiscoveryFactory.lambda$getServiceDiscovery$0(AbstractServiceDiscoveryFactory.java:53)
	at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1705)
	at org.apache.dubbo.registry.client.AbstractServiceDiscoveryFactory.getServiceDiscovery(AbstractServiceDiscoveryFactory.java:53)
	at org.apache.dubbo.admin.config.ConfigCenter.getServiceDiscoveryRegistry(ConfigCenter.java:187)
	at org.apache.dubbo.admin.config.ConfigCenter$$EnhancerBySpringCGLIB$$8706a7ed.CGLIB$getServiceDiscoveryRegistry$3(<generated>)
	at org.apache.dubbo.admin.config.ConfigCenter$$EnhancerBySpringCGLIB$$8706a7ed$$FastClassBySpringCGLIB$$c84e93b5.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244)
	at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331)
	at org.apache.dubbo.admin.config.ConfigCenter$$EnhancerBySpringCGLIB$$8706a7ed.getServiceDiscoveryRegistry(<generated>)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154)
	... 21 common frames omitted
Caused by: java.lang.IllegalStateException: failed to connect to zookeeper server
	at org.apache.dubbo.registry.zookeeper.util.CuratorFrameworkUtils.buildCuratorFramework(CuratorFrameworkUtils.java:75)
	at org.apache.dubbo.registry.zookeeper.ZookeeperServiceDiscovery.<init>(ZookeeperServiceDiscovery.java:72)
	... 36 common frames omitted

2. 更换本机的 ZooKeeper 地址就能启动成功

二、问题分析

2.1 问题原因分析

1. 应该是连接远程的 ZooKeeper 网络耗时更长，导致连接超时了

2.2 解决方向

1. 第一步 验证远程的ZooKeeper服务是否正常
2. 如果正常，则 debug 判断是否是超时配置导致未连接成功。如果超时则尝试修改Dubbo的超时配置
   1. dubbo有注册中心超时时间 dubbo.registry.timeout 是启动时注册配置中心就要用的
   2. dubbo 的 dubbo.config-center.timeout 是获取配置可能用到的

三、问题解决

3.1 确定原因

确定远程ZooKeeper是否正常
通过可视化桌面连接软件直接连接远程ZooKeeper 发现连接正常但是稍慢，那很大肯能性就是连接超时

3.2 解决过程

debug定位问题

1. 使用curator 连接Zookeeper时出问题导致抛出异常 org.apache.dubbo.registry.zookeeper.ZookeeperServiceDiscovery#ZookeeperServiceDiscovery

2. 继续定位 发现 dubbo使用curator连接ZooKeeper时，使用了 blockUntilConnected 这个方法，这里的默认超时时间是10秒 org.apache.dubbo.registry.zookeeper.util.CuratorFrameworkUtils#buildCuratorFramework
3. 这里的超时时间，默认值为 Dubbo 提前定义的枚举中获取，但是也可以从URL中获取，org.apache.dubbo.registry.zookeeper.util.CuratorFrameworkParams
4. 修改配置
   1. 【结果可行】直接修改URL admin.registry.address=zookeeper://127.0.0.1:15881?blockUntilConnectedWait=50
   2. 【结果可行】修改dubbo配置，dubbo会自动拼接到url上 dubbo.registry.timeout=50000