Soul网关源码阅读03

结合dubbo插件，发起http请求soul网关，体验dubbo代理

1、启动网关服务

1） soul-admin：启动网关管理后台，打开dubbo插件配置

2） soul-bootstrap：启动网关入口

3）启动zookeeper，使用zookeeper作为 Dubbo 服务的注册中心

zookeeper ./bin/zkServer.sh start

/usr/bin/java

ZooKeeper JMX enabled by default

Using config: /Documents/soft/zookeeper/bin/../conf/zoo.cfg

Starting zookeeper ... STARTED

4）soul-admin 编辑 Dubbo 插件配置，配置 zookeeper

{"register":"zookeeper://localhost:2181"}

5）搭建dubbo服务

soul-examples

soul-examples-apache-dubbo-service

soul-examples-dubbo-api

soul-examples-apache-dubbo-service/src/main/resources/application.yml

server:
  port: 8011
  address: 0.0.0.0
  servlet:
    context-path: /
spring:
  main:
    allow-bean-definition-overriding: true
soul:
  dubbo:
    #Soul Admin 地址
    adminUrl: http://localhost:9095 
    #设置在 Soul 网关的路由前缀，例如说 /order、/product 等等。
    contextPath: /dubbo
    #应用名。未配置情况下，默认使用 Dubbo 的应用名
    appName: dubbo  

soul-examples-apache-dubbo-service/src/main/resources/spring-dubbo.xml

在需要暴露的接口上增加 @SoulDubboClient 注解，注册到网关

@Override
@SoulDubboClient(path = "/findById", desc = "Query by Id")
public DubboTest findById(final String id) {
    DubboTest dubboTest = new DubboTest();
    dubboTest.setId(id);
    dubboTest.setName("hello world Soul Apache, findById");
    return dubboTest;
}

soul-examples-apache-dubbo-service 服务启动成功后，可以看到如下注册到网关的接口信息，并在soul-admin 插件管理后台看到如下截图。

2021-01-16 18:44:45.389  INFO 46323 --- [pool-1-thread-1] .a.d.ApacheDubboServiceBeanPostProcessor : dubbo client register success :{} {"appName":"dubbo","contextPath":"/dubbo","path":"/dubbo/findByArrayIdsAndName","pathDesc":"","rpcType":"dubbo","serviceName":"org.dromara.soul.examples.dubbo.api.service.DubboMultiParamService","methodName":"findByArrayIdsAndName","ruleName":"/dubbo/findByArrayIdsAndName","parameterTypes":"[Ljava.lang.Integer;,java.lang.String","rpcExt":"{\"timeout\":10000}","enabled":true}
2021-01-16 18:44:45.389  INFO 46323 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn          : Socket connection established, initiating session, client: /127.0.0.1:51204, server: localhost/127.0.0.1:2181
2021-01-16 18:44:45.407  INFO 46323 --- [pool-1-thread-1] .a.d.ApacheDubboServiceBeanPostProcessor : dubbo client register success :{} {"appName":"dubbo","contextPath":"/dubbo","path":"/dubbo/findByStringArray","pathDesc":"","rpcType":"dubbo","serviceName":"org.dromara.soul.examples.dubbo.api.service.DubboMultiParamService","methodName":"findByStringArray","ruleName":"/dubbo/findByStringArray","parameterTypes":"[Ljava.lang.String;","rpcExt":"{\"timeout\":10000}","enabled":true}
2021-01-16 18:44:45.430  INFO 46323 --- [pool-1-thread-1] .a.d.ApacheDubboServiceBeanPostProcessor : dubbo client register success :{} {"appName":"dubbo","contextPath":"/dubbo","path":"/dubbo/findByListId","pathDesc":"","rpcType":"dubbo","serviceName":"org.dromara.soul.examples.dubbo.api.service.DubboMultiParamService","methodName":"findByListId","ruleName":"/dubbo/findByListId","parameterTypes":"java.util.List","rpcExt":"{\"timeout\":10000}","enabled":true}
2021-01-16 18:44:45.464  INFO 46323 --- [pool-1-thread-1] .a.d.ApacheDubboServiceBeanPostProcessor : dubbo client register success :{} {"appName":"dubbo","contextPath":"/dubbo","path":"/dubbo/batchSave","pathDesc":"","rpcType":"dubbo","serviceName":"org.dromara.soul.examples.dubbo.api.service.DubboMultiParamService","methodName":"batchSave","ruleName":"/dubbo/batchSave","parameterTypes":"java.util.List","rpcExt":"{\"timeout\":10000}","enabled":true}

2、启动soul-bootstrap 服务遇到的错误

启动dubbo服务后，如果出现 dubbo client register error 注册到网关失败。

1）、可能是soul后台未配置开启dubbo插件

2）、dubbo插件配置后，需要手动重启 soul-admin 和 soul-bootstrap

3）、在网关的soul-bootstrap，pom.xml 引入对 dubbo ，nacos 的依赖

2021-01-14 15:31:30.014  INFO 6979 --- [           main] org.apache.dubbo.config.AbstractConfig   :  [DUBBO] Register dubbo service cn.iocoder.springboot.lab60.userservice.api.UserService url dubbo://192.168.2.1:20880/cn.iocoder.springboot.lab60.userservice.api.UserService?anyhost=true&application=user-service&bean.name=ServiceBean:cn.iocoder.springboot.lab60.userservice.api.UserService:1.0.0&bind.ip=192.168.2.1&bind.port=20880&deprecated=false&dubbo=2.0.2&dynamic=true&generic=false&interface=cn.iocoder.springboot.lab60.userservice.api.UserService&methods=getUser,createUser&pid=6979&qos.enable=false&release=2.7.4.1&revision=1.0.0&side=provider&timeout=1000×tamp=1610609489943&version=1.0.0 to registry registry://127.0.0.1:8848/org.apache.dubbo.registry.RegistryService?application=user-service&dubbo=2.0.2&pid=6979&qos.enable=false®istry=nacos&release=2.7.4.1×tamp=1610609489924, dubbo version: 2.7.4.1, current host: 192.168.2.1
2021-01-14 15:31:30.048  INFO 6979 --- [           main] o.a.d.qos.protocol.QosProtocolWrapper    :  [DUBBO] qos won't be started because it is disabled. Please check dubbo.application.qos.enable is configured either in system property, dubbo.properties or XML/spring-boot configuration., dubbo version: 2.7.4.1, current host: 192.168.2.1
2021-01-14 15:31:30.473 ERROR 6979 --- [pool-2-thread-1] .d.s.c.d.s.DubboServiceBeanPostProcessor : dubbo client register error :{} {"appName":"user-service","path":"/user-api/user/create","pathDesc":"创建用户","rpcType":"dubbo","serviceName":"cn.iocoder.springboot.lab60.userservice.api.UserService","methodName":"createUser","parameterTypes":"cn.iocoder.springboot.lab60.userservice.api.dto.UserCreateDTO","rpcExt":"{\"version\":\"1.0.0\"}","enabled":true}
2021-01-14 15:31:30.495 ERROR 6979 --- [pool-2-thread-1] .d.s.c.d.s.DubboServiceBeanPostProcessor : dubbo client register error :{} {"appName":"user-service","path":"/user-api/user/get","pathDesc":"获得用户详细","rpcType":"dubbo","serviceName":"cn.iocoder.springboot.lab60.userservice.api.UserService","methodName":"getUser","parameterTypes":"java.lang.Integer","rpcExt":"{\"version\":\"1.0.0\"}","enabled":true}
2021-01-14 15:31:30.855  INFO 6979 --- [           main] o.a.d.remoting.transport.AbstractServer  :  [DUBBO] Start NettyServer bind /0.0.0.0:20880, export /192.168.2.1:20880, dubbo version: 2.7.4.1, current host: 192.168.2.1
2021-01-14 15:31:30.955  INFO 6979 --- [           main] o.a.dubbo.registry.nacos.NacosRegistry   :  [DUBBO] Register: dubbo://192.168.2.1:20880/cn.iocoder.springboot.lab60.userservice.api.UserService?anyhost=true&application=user-service&bean.name=ServiceBean:cn.iocoder.springboot.lab60.userservice.api.UserService:1.0.0&deprecated=false&dubbo=2.0.2&dynamic=true&generic=false&interface=cn.iocoder.springboot.lab60.userservice.api.UserService&methods=getUser,createUser&pid=6979&release=2.7.4.1&revision=1.0.0&side=provider&timeout=1000×tamp=1610609489943&version=1.0.0, dubbo version: 2.7.4.1, current host: 192.168.2.1
2021-01-14 15:31:31.138  INFO 6979 --- [           main] o.a.dubbo.registry.nacos.NacosRegistry   :  [DUBBO] Subscribe: provider://192.168.2.1:20880/cn.iocoder.springboot.lab60.userservice.api.UserService?anyhost=true&application=user-service&bean.name=ServiceBean:cn.iocoder.springboot.lab60.userservice.api.UserService:1.0.0&bind.ip=192.168.2.1&bind.port=20880&category=configurators&check=false&deprecated=false&dubbo=2.0.2&dynamic=true&generic=false&interface=cn.iocoder.springboot.lab60.userservice.api.UserService&methods=getUser,createUser&pid=6979&qos.enable=false&release=2.7.4.1&revision=1.0.0&side=provider&timeout=1000×tamp=1610609489943&version=1.0.0, dubbo version: 2.7.4.1, current host: 192.168.2.1
2021-01-14 15:31:31.170  INFO 6979 --- [           main] o.a.dubbo.registry.nacos.NacosRegistry   :  [DUBBO] Notify urls for subscribe url provider://192.168.2.1:20880/cn.iocoder.springboot.lab60.userservice.api.UserService?anyhost=true&application=user-service&bean.name=ServiceBean:cn.iocoder.springboot.lab60.userservice.api.UserService:1.0.0&bind.ip=192.168.2.1&bind.port=20880&category=configurators&check=false&deprecated=false&dubbo=2.0.2&dynamic=true&generic=false&interface=cn.iocoder.springboot.lab60.userservice.api.UserService&methods=getUser,createUser&pid=6979&qos.enable=false&release=2.7.4.1&revision=1.0.0&side=provider&timeout=1000×tamp=1610609489943&version=1.0.0, urls: [empty://192.168.2.1:20880/cn.iocoder.springboot.lab60.userservice.api.UserService?anyhost=true&application=user-service&bean.name=ServiceBean:cn.iocoder.springboot.lab60.userservice.api.UserService:1.0.0&bind.ip=192.168.2.1&bind.port=20880&category=providers&check=false&deprecated=false&dubbo=2.0.2&dynamic=true&generic=false&interface=cn.iocoder.springboot.lab60.userservice.api.UserService&methods=getUser,createUser&pid=6979&qos.enable=false&release=2.7.4.1&revision=1.0.0&side=provider&timeout=1000×tamp=1610609489943&version=1.0.0], dubbo version: 2.7.4.1, current host: 192.168.2.1
2021-01-14 15:31:31.172  INFO 6979 --- [client.listener] o.a.dubbo.registry.nacos.NacosRegistry   :  [DUBBO] Notify urls for subscribe url provider://192.168.2.1:20880/cn.iocoder.springboot.lab60.userservice.api.UserService?anyhost=true&application=user-service&bean.name=ServiceBean:cn.iocoder.springboot.lab60.userservice.api.UserService:1.0.0&bind.ip=192.168.2.1&bind.port=20880&category=configurators&check=false&deprecated=false&dubbo=2.0.2&dynamic=true&generic=false&interface=cn.iocoder.springboot.lab60.userservice.api.UserService&methods=getUser,createUser&pid=6979&qos.enable=false&release=2.7.4.1&revision=1.0.0&side=provider&timeout=1000×tamp=1610609489943&version=1.0.0, urls: [empty://192.168.2.1:20880/cn.iocoder.springboot.lab60.userservice.api.UserService?anyhost=true&application=user-service&bean.name=ServiceBean:cn.iocoder.springboot.lab60.userservice.api.UserService:1.0.0&bind.ip=192.168.2.1&bind.port=20880&category=providers&check=false&deprecated=false&dubbo=2.0.2&dynamic=true&generic=false&interface=cn.iocoder.springboot.lab60.userservice.api.UserService&methods=getUser,createUser&pid=6979&qos.enable=false&release=2.7.4.1&revision=1.0.0&side=provider&timeout=1000×tamp=1610609489943&version=1.0.0], dubbo version: 2.7.4.1, current host: 192.168.2.1
2021-01-14 15:31:31.176  INFO 6979 --- [           main] c.i.s.l.u.UserServiceApplication         : Started UserServiceApplication in 3.075 seconds (JVM running for 4.349)
2021-01-14 15:31:31.179  INFO 6979 --- [pool-1-thread-1] .b.c.e.AwaitingNonWebApplicationListener :  [Dubbo] Current Spring Boot Application is await...

4）、开启dubbo插件
soul-bootstrap -> pom.xml 中默认配置的是 alibaba dubbo 依赖，如果启用 apache dubbo 依赖，注意要注释掉 alibaba dubbo 依赖，否则启动会报错

Duplicate key org.dromara.soul.plugin.apache.dubbo.handler.ApacheDubboPluginDataHandler@234a8f27

Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.dromara.soul.sync.data.api.PluginDataSubscriber]: Factory method 'pluginDataSubscriber' threw exception; nested exception is java.lang.IllegalStateException: Duplicate key org.dromara.soul.plugin.apache.dubbo.handler.ApacheDubboPluginDataHandler@234a8f27
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185) ~[spring-beans-5.2.2.RELEASE.jar:5.2.2.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:651) ~[spring-beans-5.2.2.RELEASE.jar:5.2.2.RELEASE]
    ... 43 common frames omitted
Caused by: java.lang.IllegalStateException: Duplicate key org.dromara.soul.plugin.apache.dubbo.handler.ApacheDubboPluginDataHandler@234a8f27

3、源码解析-dubbo 接口是如何注册到网关的呢

1）dubbo 接口是如何注册到网关的，我们在dubbo服务中引入了soul网关的依赖，并在接口上增加了@SoulDubboClient 注解。

    org.dromara
    soul-spring-boot-starter-client-apache-dubbo
    ${soul.version}

2）来看看 soul-spring-boot-starter-client-apache-dubbo 源码

soul-spring-boot-starter-client-apache-dubbo 使用了自动装配，说明在服务启动后会扫描@SoulDubboClient 注解，并将标注的接口注册到网关。

// soul-spring-boot-starter-client-apache-dubbo/src/main/resources/META-INF/spring.factories


org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
org.dromara.soul.springboot.starter.client.apache.dubbo.SoulApacheDubboClientConfiguration

3）SoulApacheDubboClientConfiguration 源码
实例化了 DubboConfig，读取网关配置信息，例如：网关地址，服务路由等
实例化了 ApacheDubboServiceBeanPostProcessor Dubbo服务后置处理器，此类应该跟向网关注册dubbo服务接口有关。

/**
 * The type Soul apache dubbo client configuration.
 * @author xiaoyu
 */
@Configuration
public class SoulApacheDubboClientConfiguration {
    /**
     * Apache dubbo service bean post processor alibaba dubbo service bean post processor.
     * @param dubboConfig the dubbo config
     * @return the alibaba dubbo service bean post processor
     */
    @Bean
    public ApacheDubboServiceBeanPostProcessor apacheDubboServiceBeanPostProcessor(final DubboConfig dubboConfig) {
        return new ApacheDubboServiceBeanPostProcessor(dubboConfig);
    }
    /**
     * Dubbo config dubbo config.
     * @return the dubbo config
     */
    @Bean
    @ConfigurationProperties(prefix = "soul.dubbo")
    public DubboConfig dubboConfig() {
        return new DubboConfig();
    }
}

4）ApacheDubboServiceBeanPostProcessor ，Apache Dubbo ServiceBean后处理器。

ApacheDubboServiceBeanPostProcessor 初始化会读取跟dubbo配置相关的网关配置，例如网关soul-admin后台地址
还初始化了一个 ThreadPoolExecutor线程池，应该是通过线程池来并发的向网关注册 dubbo接口地址，来提高接口注册的效率。

public ApacheDubboServiceBeanPostProcessor(final DubboConfig dubboConfig) {
    String contextPath = dubboConfig.getContextPath();
    String adminUrl = dubboConfig.getAdminUrl();
    if (StringUtils.isEmpty(contextPath)
            || StringUtils.isEmpty(adminUrl)) {
        throw new RuntimeException("apache dubbo client must config the contextPath, adminUrl");
    }
    this.dubboConfig = dubboConfig;
    url = dubboConfig.getAdminUrl() + "/soul-client/dubbo-register";
    executorService = new ThreadPoolExecutor(1, 1, 0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<>());
}

5）这里通过线程池来向网关并发的注册接口

@Override
public void onApplicationEvent(final ContextRefreshedEvent contextRefreshedEvent) {
    if (Objects.nonNull(contextRefreshedEvent.getApplicationContext().getParent())) {
        return;
    }
    // Fix bug(https://github.com/dromara/soul/issues/415), upload dubbo metadata on ContextRefreshedEvent
    Map serviceBean = contextRefreshedEvent.getApplicationContext().getBeansOfType(ServiceBean.class);
    for (Map.Entry entry : serviceBean.entrySet()) {
        executorService.execute(() -> handler(entry.getValue()));
    }
}

6）通过反射来读取注解：@SoulDubboClient
RegisterUtils.doRegister(); 应该是用来请求网关soul-admin发起请求的

private void handler(final ServiceBean serviceBean) {
    Class clazz = serviceBean.getRef().getClass();
    if (ClassUtils.isCglibProxyClass(clazz)) {
        String superClassName = clazz.getGenericSuperclass().getTypeName();
        try {
            clazz = Class.forName(superClassName);
        } catch (ClassNotFoundException e) {
            log.error(String.format("class not found: %s", superClassName));
            return;
        }
    }
    final Method[] methods = ReflectionUtils.getUniqueDeclaredMethods(clazz);
    for (Method method : methods) {
        SoulDubboClient soulDubboClient = method.getAnnotation(SoulDubboClient.class);
        if (Objects.nonNull(soulDubboClient)) {
            RegisterUtils.doRegister(buildJsonParams(serviceBean, soulDubboClient, method), url, RpcTypeEnum.DUBBO);
        }
    }
}

7）来看看 RegisterUtils.doRegister()
果然通过下面代码可知，此处发起了最终的接口注册调用，并且注册的结果信息也此打印。

/**
 * RegisterUtils.
 * @author severez
 */
@Slf4j
public final class RegisterUtils {
    /**
     * call register api.
     * @param json        request body
     * @param url         url
     * @param rpcTypeEnum rcp type
     */
    public static void doRegister(final String json, final String url, final RpcTypeEnum rpcTypeEnum) {
        try {
            String result = OkHttpTools.getInstance().post(url, json);
            if (AdminConstants.SUCCESS.equals(result)) {
                log.info("{} client register success: {} ", rpcTypeEnum.getName(), json);
            } else {
                log.error("{} client register error: {} ", rpcTypeEnum.getName(), json);
            }
        } catch (IOException e) {
            log.error("cannot register soul admin param, url: {}, request body: {}", url, json, e);
        }
    }
}

4、dubbo 接口注册到 soul-admin 原理思维导图：

5、总结：

在上一篇的divde插件源码阅读中，整个流程比较混乱，把自己绕晕了，没有抓住重点。今天从单独从
soul-spring-boot-starter-client-apache-dubbo入手，一步步看清楚了dubbo 接口是如何注册到soul-admin的。
而且原理相对简单，思路清晰。divde 插件接口注册应该也是类似原理。