dubbo之Protocol(协议)

前言

前面我们讲到的服务暴露流程中,没有涉及Protocol相关操作,本篇我们来看Protocol的实现。 本文将重点分析Protocol的核心功能,以及Protocol的设计与实现原理。从Protocol接口开始,逐个分析具体Protocol的实现。Protocol是dubbo中协议的抽象,负责服务的暴露、引用,在dubbo整个框架设计中位于protocol层(远程调用层),在exchange层(信息交换层)之上(各层间依赖关系:protocol -> exchange -> transport -> serialize)。Protocol接口支持SPI扩展,默认SPI实现是DubboProtocol,同时支持方法级SPI。接口核心方法包括export(服务暴露)、refer(服务引用)、destroy(协议销毁)。AbstractProtocol作为基类实现,提供默认destroy方法实现。方便理解起见,我把Protocol的SPI扩展实现分为两种,第一种直接实现Protocol接口,这里暂且称之为Protocol代理,包括QosProtocolWrapper、RegistryProtocol、ProtocolFilterWrapper、ProtocolListenerWrapper;第二种继承AbstractProtocol基类,具体实现export、refer方法功能,最具代表性的是DubboProtocol,但是RedisProtocol、MemcachedProtocol、MockProtocol仅提供服务引用功能(仅实现refer接口),不支持服务暴露;另外,有一部分Protocol通过继承AbstractProxyProtocol(当然,AbstractProxyProtocol继承了AbstractProtocol)实现,AbstractProxyProtocol内部引入ProxyFactory,借助Hessian、Spring的httpInvoker、Spring的RMI、apache的cxf(webService框架)等Rpc、RMI、Service框架实现服务的暴露和引用。类之间的继承关系如下

Protocol-UML (2).jpg

一、Protocol接口

上面提到,Protocol对外暴露的核心接口有export(服务暴露)、refer(服务引用)、destroy(协议销毁),dubbo默认采用的协议是DubboProtocol,其中服务暴露和服务引用支持方法级SPI。核心方法介绍如下:

1.1、 服务暴露

服务暴露,即对外暴露可用服务,其本质是创建socket连接,并绑定url,对外暴露服务端口

export方法有以下几点需要注意
1、收到请求后,会记录请求源地址到 RpcContext,RpcContext.getContext().setRemoteAddress();
2、export方法要求幂等,即对于同一个URL来说调用一次与调用n次的结果一致;
3、参数Invoker由框架传递,protocol无需关注。

1.2、服务引用

refer方法有以下几点需要注意
1、用户调用refer方法返回的Invoker对象的invoke方法时,protocol需要相应的执行Invoker的invoke方法;
2、由protocol负责refer方法返回的Invoker的具体实现;
3、当URL中设置了check=false时,连接失败时protocol应该尝试恢复而不是抛异常。

1.3 、协议销毁

destroy方法有以下几点需要注意
1、销毁方法需要取消当前协议暴露和引用的所有服务;
2、释放所有占用的资源,比如连接、端口等;
3、当被销毁之后,protocol可以继续暴露或者引用新服务。

一、Protocol代理

Protocol的代理实现主要有QosProtocolWrapper、ProtocolFilterWrapper、ProtocolListenerWrapper、RegistryProtocol。其中OosProtocolWrapper提供服务的Qos,即为dubbo服务提供Qos保障(Qos 指利用各种基础技术,为指定的网络通信提供更好的服务能力,是网络的一种安全机制, 用来解决网络延迟和阻塞等问题,通常Oos的关键指标包括可用性、吞吐量、时延、时延变化(包括抖动和漂移)和丢失)。严格来讲,RegistryProtocol并非Protocol的代理实现(dubbo中对代理类的定义是,构造方法有且仅有一个参数,即被代理类对象),但RegistryProtocol的服务暴露、引用均由内部的protocol引用实现,所以将其归类到代理,一并进行解析。

1、QosProtocolWrapper

QosProtocolWrapper是Protocol的代理实现,提供服务的Qos保障,需要注意的是,dubbo默认开启Qos,即默认qos.enable=true;另外,只有协议采用registry时,才会启动Qos;Qos的实现借助Netty,打开本地端口(*默认端口22222*),创建并启动Server。支持telnet使用,格式:telnet ip port。
@Override
public  Exporter export(Invoker invoker) throws RpcException {
    // protocol使用registry时,才会启动Qos服务,否则直接使用代理的protocol实现服务暴露
    if (Constants.REGISTRY_PROTOCOL.equals(invoker.getUrl().getProtocol())) {
        startQosServer(invoker.getUrl());
        return protocol.export(invoker);
    }
    return protocol.export(invoker);
}

@Override
public  Invoker refer(Class type, URL url) throws RpcException {
    // protocol使用registry时,才会启动Qos,复杂直接使用代理的protocol实现服务引用
    if (Constants.REGISTRY_PROTOCOL.equals(url.getProtocol())) {
        startQosServer(url);
        return protocol.refer(type, url);
    }
    return protocol.refer(type, url);
}

来看Qos服务启动的逻辑,核心逻辑是创建Server并启动(这里的Server是个单独的工具类),比较简单,直接来看Server的启动逻辑

public void start() throws Throwable {
    if (!started.compareAndSet(false, true)) {
        return;
    }
    boss = new NioEventLoopGroup(0, new DefaultThreadFactory("qos-boss", true));
    worker = new NioEventLoopGroup(0, new DefaultThreadFactory("qos-worker", true));
    // 创建NettyServer绑定端口
    ServerBootstrap serverBootstrap = new ServerBootstrap();
    serverBootstrap.group(boss, worker);
    serverBootstrap.channel(NioServerSocketChannel.class);
    serverBootstrap.childOption(ChannelOption.TCP_NODELAY, true);
    serverBootstrap.childOption(ChannelOption.SO_REUSEADDR, true);
    serverBootstrap.childHandler(new ChannelInitializer() {
        @Override
        protected void initChannel(Channel ch) throws Exception {
            ch.pipeline().addLast(new QosProcessHandler(welcome, acceptForeignIp));
        }
    });
    try {
        serverBootstrap.bind(port).sync();
        logger.info("qos-server bind localhost:" + port);
    } catch (Throwable throwable) {
        logger.error("qos-server can not bind localhost:" + port, throwable);
        throw throwable;
    }
}

2、ProtocolFilterWrapper

ProtocolFilterWrapper的核心逻辑就是buildInvokerChain,用于构建invoker的调用链,前面介绍buildInvokerChain方法时,做过相关介绍,这里不过多解析。

3、ProtocolListenerWrapper

ProtocolListenerWrapper同样是Protocol的代理实现,借助ExporterListener(支持SPI扩展,无实质逻辑)、InvokerListener(支持SPI扩展,无实质逻辑),用于提供服务暴露、引用实例的代理,只有protocol非registry时才会返回实例代理。仅需关注核心方法export、refer,逻辑也比较简单

@Override
public  Exporter export(Invoker invoker) throws RpcException {
    // 协议值为registry时,直接执行服务暴露,否则返回服务暴露后的代理实例
    if (Constants.REGISTRY_PROTOCOL.equals(invoker.getUrl().getProtocol())) {
        return protocol.export(invoker);
    }
    return new ListenerExporterWrapper(protocol.export(invoker),
            Collections.unmodifiableList(ExtensionLoader.getExtensionLoader(ExporterListener.class)
                    .getActivateExtension(invoker.getUrl(), Constants.EXPORTER_LISTENER_KEY)));
}

@Override
public  Invoker refer(Class type, URL url) throws RpcException {
    // 协议值为registry时,直接执行服务的引用,否则返回的代理实例
    if (Constants.REGISTRY_PROTOCOL.equals(url.getProtocol())) {
        return protocol.refer(type, url);
    }
    return new ListenerInvokerWrapper(protocol.refer(type, url),
            Collections.unmodifiableList(
                    ExtensionLoader.getExtensionLoader(InvokerListener.class)
                            .getActivateExtension(url, Constants.INVOKER_LISTENER_KEY)));
}

上面的代码中,ListenerExporterWrapper与ListenerInvokerWrapper的逻辑比较简单,构造方法中分别执行ExporterListener.export、InvokerListener.invoke方法的代理逻辑。

4、RegistryProtocol

RegistryProtocol,即注册中心协议实现,服务暴露多了创建注册中心和服务注册逻辑。当URL中protocol=registry时,采用该协议。分析RegisgtryProtocol过程中,Registry的实现我们以全部以Registry的默认SPI实现即DubboRegistry为例(后续会对Registry专门开篇解析)。RegistryProtocol的服务暴露包括:Url构建、服务本地暴露、注册中心构建、服务注册几个步骤

4.1 服务暴露

1、构建URL

第一步,构建registryUrl、providerUrl、overrideSubscribeUrl,以当前Invoker中的URL(暂且叫做originUrl)为参照,按照原型模式创建。registryUrl,即将originUrl中procotol值替换为registry值,从下面的示例可以清楚看出URL构建的过程及结果。

// 当前Invoker中url值
originUrl = "registry://127.0.0.1:9090?export=dubbo://127.0.0.1:9453/org.apache.dubbo.registry.protocol.DemoService:1.0.0?notify=true&methods=test1,test2&side=con&side=consumer"

// registryUrl,即将protocol值由registry变更为dubbo(默认注册中心实现DubboRegistry)
registryUrl = "dubbo://127.0.0.1:9090?export=dubbo://127.0.0.1:9453/org.apache.dubbo.registry.protocol.DemoService:1.0.0?notify=true&methods=test1,test2&side=con&side=consumer"

//  providerUrl,即originUrl中export值
providerUrl = "dubbo://127.0.0.1:9453/org.apache.dubbo.registry.protocol.DemoService:1.0.0?methods=test1,test2¬ify=true&side=consumer"

// overrideSubscribeUrl,即将originUrl中protocol值替换为provider,
//并新增category参数,值为configurators,为后面providerUrl的参数融合做准备
overrideSubscribeUrl = "provider://127.0.0.1:9453/org.apache.dubbo.registry.protocol.DemoService:1.0.0?category=configurators&check=false&methods=test1,test2¬ify=true&side=consumer" 

完成相关URL构建之后,进行URL参数融合,核心代码如下,重点关注overrideUrlWithConfig方法。providerConfigurationListener、serviceConfigurationListener构建过程中,会加载动态配置(关于配置的解析详见dubbo之配置(Configuration))到对应listener的configurators,通过listener的overrideUrl方法将configurators配置融合到providerUrl参数

// 构建信息融合listener
final OverrideListener overrideSubscribeListener = new OverrideListener(overrideSubscribeUrl, originInvoker);
overrideListeners.put(overrideSubscribeUrl, overrideSubscribeListener);
// 动态配置信息融合到provider
providerUrl = overrideUrlWithConfig(providerUrl, overrideSubscribeListener);

// 配置信息融合
private URL overrideUrlWithConfig(URL providerUrl, OverrideListener listener) {
            // 应用级配置信息融合到providerUrl
        providerUrl = providerConfigurationListener.overrideUrl(providerUrl);
        ServiceConfigurationListener serviceConfigurationListener = new ServiceConfigurationListener(providerUrl, listener);
            // 服务级配置信息融合到providerUrl
        serviceConfigurationListeners.put(providerUrl.getServiceKey(), serviceConfigurationListener);
        return serviceConfigurationListener.overrideUrl(providerUrl);
    }

有一点需要注意,ServiceConfigurationListener初始化过程中会从动态配置拉取最新配置信息,然后执行reExport(重新暴露),入口在process方法,逻辑如下:

protected final void initWith(String key) {
    //动态配置
    DynamicConfiguration dynamicConfiguration = DynamicConfiguration.getDynamicConfiguration();
    // 如果是nopDynamicConfiguration,那么这里什么都不会做
    dynamicConfiguration.addListener(key, this);
    String rawConfig = dynamicConfiguration.getConfig(key);
    if (!StringUtils.isEmpty(rawConfig)) {
        //直接根据字符串配置,赋值configurators.并执行notifyOverride模板方法
        process(new ConfigChangeEvent(key, rawConfig));
    }
}

configurators的初始化不再过多解析,重点关注process方法中调用的notifyOverrides模板方法,以ProviderConfigurationListener为例,notifyOverrides实现如下:

@Override
protected void notifyOverrides() {
    overrideListeners.values().forEach(listener -> ((OverrideListener) listener).doOverrideIfNecessary());
}

继续看方法内部,执行OverrideListener的doOverrideIfNecessary方法,逻辑比较简单,对比配置覆盖前后的url,若不一致,则重新暴露

public synchronized void doOverrideIfNecessary() {
    final Invoker invoker;
    // 原始invoker或者invoker代理
    if (originInvoker instanceof InvokerDelegate) {
        invoker = ((InvokerDelegate) originInvoker).getInvoker();
    } else {
        invoker = originInvoker;
    }
    //The origin invoker,从Invoker中解析出原始URL(非注册中心url)
    URL originUrl = RegistryProtocol.this.getProviderUrl(invoker);
    String key = getCacheKey(originInvoker);
    ExporterChangeableWrapper exporter = bounds.get(key);
    if (exporter == null) {
        logger.warn(new IllegalStateException("error state, exporter should not be null"));
        return;
    }
    //The current, may have been merged many times
    URL currentUrl = exporter.getInvoker().getUrl();
    //Merged with this configuration,所有configurator参数合并到原始URL,生成新url
    URL newUrl = getConfigedInvokerUrl(configurators, originUrl);
    newUrl = getConfigedInvokerUrl(serviceConfigurationListeners.get(originUrl.getServiceKey())
            .getConfigurators(), newUrl);
    newUrl = getConfigedInvokerUrl(providerConfigurationListener.getConfigurators(), newUrl);
    // 如果当前url与合并后的新url不一致,那么,重新暴露新的url
    if (!currentUrl.equals(newUrl)) {
        RegistryProtocol.this.reExport(originInvoker, newUrl);
        logger.info("exported provider url changed, origin url: " + originUrl +
                ", old export url: " + currentUrl + ", new export url: " + newUrl);
    }
}

重新暴露即reExport方法分为两部分,服务重新暴露和服务重新注册

public  void reExport(final Invoker originInvoker, URL newInvokerUrl) {
    // update local exporter,本地exporter更新,检查本地Exporter,通过代理protocol重新暴露
    ExporterChangeableWrapper exporter = doChangeLocalExport(originInvoker, newInvokerUrl);
    // update registry
    URL registryUrl = getRegistryUrl(originInvoker);
    final URL registeredProviderUrl = getRegisteredProviderUrl(newInvokerUrl, registryUrl);

    //decide if we need to re-publish
    ProviderInvokerWrapper providerInvokerWrapper = ProviderConsumerRegTable.getProviderWrapper(registeredProviderUrl, originInvoker);
    ProviderInvokerWrapper newProviderInvokerWrapper = ProviderConsumerRegTable.registerProvider(originInvoker, registryUrl, registeredProviderUrl);
    /**
     * Only if the new url going to Registry is different with the previous one should we do unregister and register.
     * 原始provider已注册,且原始providerUrl与新的providerUrl不一致,执行重新注册操作(注销、重新注册)
     */
    if (providerInvokerWrapper.isReg() && !registeredProviderUrl.equals(providerInvokerWrapper.getProviderUrl())) {
        unregister(registryUrl, providerInvokerWrapper.getProviderUrl());
        register(registryUrl, registeredProviderUrl);
        newProviderInvokerWrapper.setReg(true);
    }
    //更新注册的url
    exporter.setRegisterUrl(registeredProviderUrl);
}

2、服务本地暴露

服务本地暴露,先从缓存中取之前的暴露结果,取通过protocol引用执行服务暴露。这里实际执行的是RegistryProtocol中通过IOC注入的protocol(默认是DubboProtocol),也就是说实际执行的是DubboProtocol的export,执行完成之后,包装结果Invoker到ExporterChangeableWrapper,并返回。来看代码:

private  ExporterChangeableWrapper doLocalExport(final Invoker originInvoker, URL providerUrl) {
    String key = getCacheKey(originInvoker);
    ExporterChangeableWrapper exporter = (ExporterChangeableWrapper) bounds.get(key);
    if (exporter == null) {
        synchronized (bounds) {
            exporter = (ExporterChangeableWrapper) bounds.get(key);
            if (exporter == null) {
                //本地暴露,即暴露invoker并缓存到bounds
                final Invoker invokerDelegete = new InvokerDelegate(originInvoker, providerUrl);
                // 生成代理类exporter
                exporter = new ExporterChangeableWrapper((Exporter) protocol.export(invokerDelegete), originInvoker);
                bounds.put(key, exporter);
            }
        }
    }
    return exporter;
}

3、构建注册中心

注册中心通过RegistryFactory构建,简单介绍一下RegistryFactory,RegistryFactory接口支持SPI,默认SPI实现是DubboRegistryFactory,也就是说除非URL中指定了registry值,否则默认采用DubboRegistry。我们以DubboRegistry为例来解析注册中心的构建过程:

private Registry getRegistry(final Invoker originInvoker) {
    URL registryUrl = getRegistryUrl(originInvoker);
    return registryFactory.getRegistry(registryUrl);
}

getRegistry方法由RegistryFactory的基类AbstractRegistryFactory实现,基类内部定义了模板方法createRegistry,由子类实现。直接来看DubboRegistryFactory的createRegistry逻辑。基类方法getRegistry中,dubbo会将当前registryUrl中的path参数替换成org.apache.dubbo.registry.RegistryService,同时补全methods参数值,即RegistryService声明的方法lookup,unsubscribe,subscribe,unregister,register。那么创建Registry时,url参数值为:

dubbo://127.0.0.1:9090/org.apache.dubbo.registry.RegistryService?callbacks=10000&connect.timeout=10000&interface=org.apache.dubbo.registry.RegistryService&lazy=true&methods=lookup,unsubscribe,subscribe,unregister,register&reconnect=false&sticky=true&subscribe.1.callback=true&timeout=10000&unsubscribe.1.callback=false

接着,根据backup参数值,url被拆分为多个url存放于url列表,列表中url除地址外,其他参数完全相同。然后,借助RegistryDirectory,完成注册中心的创建

@Override
public Registry createRegistry(URL url) {
    //  补全RegistryService相关参数
    url = getRegistryURL(url);
    List urls = new ArrayList<>();
    urls.add(url.removeParameter(Constants.BACKUP_KEY));
    String backup = url.getParameter(Constants.BACKUP_KEY);
    // 根据backUp地址,拆分成多个url,url列表作为RegistryDirectory的notify参数
    if (backup != null && backup.length() > 0) {
        String[] addresses = Constants.COMMA_SPLIT_PATTERN.split(backup);
        for (String address : addresses) {
            urls.add(url.setAddress(address));
        }
    }
    // 创建RegistryDirectory
    RegistryDirectory directory = new RegistryDirectory<>(RegistryService.class, url.addParameter(Constants.INTERFACE_KEY, RegistryService.class.getName()).addParameterAndEncoded(Constants.REFER_KEY, url.toParameterString()));
    // join生成虚拟Invoke
    Invoker registryInvoker = cluster.join(directory);
    // 创建RegistryService代理对象
    RegistryService registryService = proxyFactory.getProxy(registryInvoker);
    // 创建注册中心
    DubboRegistry registry = new DubboRegistry(registryInvoker, registryService);
    directory.setRegistry(registry);
    // 这里的protocol就是通过ExtensionLoader的spi注入的
    directory.setProtocol(protocol);
    directory.setRouterChain(RouterChain.buildChain(url));
    // 刷新Directory内invokers缓存
    directory.notify(urls);
    // 订阅consumer数据,刷新内部configurators,调用registry代理的subscribe()
    directory.subscribe(new URL(Constants.CONSUMER_PROTOCOL, NetUtils.getLocalHost(), 0, RegistryService.class.getName(), url.getParameters()));
    return registry;
}

4、服务注册

注册中心创建完成后,执行服务注册,逻辑比较简单,直接看代码

// 注册中心服务URL与服务提供者URL参数合并.
final URL registeredProviderUrl = getRegisteredProviderUrl(providerUrl, registryUrl);
// 服务提供者注册到注册表
ProviderInvokerWrapper providerInvokerWrapper = ProviderConsumerRegTable.registerProvider(originInvoker,
        registryUrl, registeredProviderUrl);
//to judge if we need to delay publish,服务是否需要注册发布,默认true
boolean register = registeredProviderUrl.getParameter("register", true);
if (register) {
    // 执行服务注册,核心逻辑即执行registry的register,以DubboRegistry为例,将服务url放入registered缓存,然后再执行RegistryService代理对象的register方法。
    register(registryUrl, registeredProviderUrl);
    providerInvokerWrapper.setReg(true);
}
// Deprecated! Subscribe to override rules in 2.6.x or before.
registry.subscribe(overrideSubscribeUrl, overrideSubscribeListener);

最后,创建DestroyableExporter,并返回。

4.2 服务引用

服务引用过程包括创建注册中心、创建注册中心目录(RegistryDirectory)、聚合Invoker几部分

public  Invoker refer(Class type, URL url) throws RpcException {
    url = url.setProtocol(url.getParameter(REGISTRY_KEY, DEFAULT_REGISTRY)).removeParameter(REGISTRY_KEY);
    Registry registry = registryFactory.getRegistry(url);
    // type为RegistryService,则直接返回当前Invoker代理
    if (RegistryService.class.equals(type)) {
        return proxyFactory.getInvoker((T) registry, type, url);
    }

    // group="a,b" or group="*",多个group情况下,使用mergableCluster
    Map qs = StringUtils.parseQueryString(url.getParameterAndDecoded(REFER_KEY));
    String group = qs.get(Constants.GROUP_KEY);
    if (group != null && group.length() > 0) {
        if ((COMMA_SPLIT_PATTERN.split(group)).length > 1 || "*".equals(group)) {
            return doRefer(getMergeableCluster(), registry, type, url);
        }
    }
    // 真正的引用逻辑
    return doRefer(cluster, registry, type, url);
}
private  Invoker doRefer(Cluster cluster, Registry registry, Class type, URL url) {
    // 创建RegistryDirectory
    RegistryDirectory directory = new RegistryDirectory(type, url);
    directory.setRegistry(registry);
    directory.setProtocol(protocol);
    // all attributes of REFER_KEY
    Map parameters = new HashMap(directory.getUrl().getParameters());
    URL subscribeUrl = new URL(CONSUMER_PROTOCOL, parameters.remove(REGISTER_IP_KEY), 0, type.getName(), parameters);
    if (!ANY_VALUE.equals(url.getServiceInterface()) && url.getParameter(REGISTER_KEY, true)) {
        directory.setRegisteredConsumerUrl(getRegisteredConsumerUrl(subscribeUrl, url));
        // 这里为什么会有服务的注册?看代码日志,是为了解决issue#3295,具体什么问题已经看不到了,属实不理解
        registry.register(directory.getRegisteredConsumerUrl());
    }
    directory.buildRouterChain(subscribeUrl);
    directory.subscribe(subscribeUrl.addParameter(CATEGORY_KEY,
            PROVIDERS_CATEGORY + "," + CONFIGURATORS_CATEGORY + "," + ROUTERS_CATEGORY));

    // cluster聚合,
    Invoker invoker = cluster.join(directory);
    // 注册表注册
    ProviderConsumerRegTable.registerConsumer(invoker, url, subscribeUrl, directory);
    return invoker;
}

这里有个问题需要注意,在doRefer方法中有这么一段代码,git log显示是为了解决issue#3295做的修改,看不到3295具体是什么问题,如果有知道的朋友,劳烦告知,我及时修正说明。

if (!ANY_VALUE.equals(url.getServiceInterface()) && url.getParameter(REGISTER_KEY, true)) {
        directory.setRegisteredConsumerUrl(getRegisteredConsumerUrl(subscribeUrl, url));
        // 这里为什么会有服务的注册?看代码日志,是为了解决issue#3295,具体什么问题已经看不到了,属实不理解
        registry.register(directory.getRegisteredConsumerUrl());
}

4.3、协议销毁

销毁比较容易理解,注销所有exporter、清理所有exporter缓存、移除所有配置监听器。

@Override
public void destroy() {
    // 注销所有exporter
    List> exporters = new ArrayList>(bounds.values());
    for (Exporter exporter : exporters) {
        exporter.unexport();
    }
    // 清除所有exporter缓存
    bounds.clear();
    // 移除所有监听器
    DynamicConfiguration.getDynamicConfiguration()
            .removeListener(ApplicationModel.getApplication() + CONFIGURATORS_SUFFIX, providerConfigurationListener);
}

二、Protocol实现

Protocol的直接实现类包括常见的DubboProtocol、HessianProtocol等。下面从AbstractProtocol开始,逐个分析.

1、AbstractProtocol

AbstractProtocol基类并没有对Protocol接口做过多实现,仅实现了destroy方法,逻辑也比较简单,销毁所有的Exporter和Invoker,具体代码如下:

@Override
public void destroy() {
    // 注意顺序:先销毁所有invoker,再销毁所有exporter
    for (Invoker invoker : invokers) {
        if (invoker != null) {
            invokers.remove(invoker);
            // 中间日志打印省略
            invoker.destroy();
            } catch (Throwable t) {
                logger.warn(t.getMessage(), t);
            }
        }
    }
    for (String key : new ArrayList(exporterMap.keySet())) {
        Exporter exporter = exporterMap.remove(key);
        if (exporter != null) {
                    // 日志打印省略
                exporter.unexport();
            } catch (Throwable t) {
                logger.warn(t.getMessage(), t);
            }
        }
    }
}

2、DubboProtocol

DubboProtocol是Protocol的默认SPI实现,默认端口20880;除了transporter依赖netty之外,其他所有逻辑都是dubbo自己实现。开文提到,protocol位于exchange层之上(protocol直接依赖exchange),所以这里也会对exchange层的部分实现做相关分析。下面,我们仍以export、refer、destroy方法为入口,开始解析DubboProtocol。

2.1 export(服务暴露)

先来看export方法,核心逻辑分为两部分(stubService的逻辑处理比较简单),创建Exporter并缓存、绑定并启动NettyServer(实际上是绑定ip端口,建立socket连接);exporter的创建和缓存比较简单:

// export service.创建exporter并缓存
String key = serviceKey(url);
DubboExporter exporter = new DubboExporter(invoker, key, exporterMap);
exporterMap.put(key, exporter);
// 对stubService的处理省略

// 绑定server
openServer(url);
// 无实质逻辑,可以忽略
optimizeSerialization(url);

重点关注openServer方法。首先,dubbo会先从serverMap缓存里取server实例,取到则会执行reset操作,具体逻辑在HeaderExchangeServer#reset方法;取不到则直接创建,核心逻辑在createServer方法。createServer方法内部有几个细节需要注意: 1、server关闭发送只读事件,开关默认开启;2、默认心跳间隔60s;3、默认采用NettyTransporter;4、编解码器Codec,默认采用DubboCodec。createServer方法实际上通过Exchanger接口的bind方法实现,以HeaderExchanger(Exchanger的默认SPI实现)为例,核心逻辑如下:

@Override
public ExchangeServer bind(URL url, ExchangeHandler handler) throws RemotingException {
    return new HeaderExchangeServer(Transporters.bind(url, new DecodeHandler(new HeaderExchangeHandler(handler))));
}

可以看到,bind方法又借助Transporter的bind实现,创建Server实例。以NettyTransporter(Transporter接口的默认SPI实现)为例,直接创建NettyServer:

@Override
    public Server bind(URL url, ChannelHandler listener) throws RemotingException {
        return new NettyServer(url, listener);
    }

Server实例化逻辑大部分由父类AbstractServer构造方法完成。另外,AbstractServer定义模板方法doOpen,并在其构造方法中调用,doOpen方法逻辑由子类Server实现,也就说在实例化NettyServer时会执行NettyServer.doOpen的动作,具体代码如下:

public AbstractServer(URL url, ChannelHandler handler) throws RemotingException {
  // 参数初始化阶段
  super(url, handler);
    localAddress = getUrl().toInetSocketAddress();
    String bindIp = getUrl().getParameter(Constants.BIND_IP_KEY, getUrl().getHost());
    int bindPort = getUrl().getParameter(Constants.BIND_PORT_KEY, getUrl().getPort());
    if (url.getParameter(Constants.ANYHOST_KEY, false) || NetUtils.isInvalidLocalHost(bindIp)) {
        bindIp = Constants.ANYHOST_VALUE;
    }
    bindAddress = new InetSocketAddress(bindIp, bindPort);
    this.accepts = url.getParameter(Constants.ACCEPTS_KEY, Constants.DEFAULT_ACCEPTS);
    this.idleTimeout = url.getParameter(Constants.IDLE_TIMEOUT_KEY, Constants.DEFAULT_IDLE_TIMEOUT);
  // 执行doOpen模板方法,由具体的Server实现  
  try {
        doOpen();
        if (logger.isInfoEnabled()) {
            logger.info("Start " + getClass().getSimpleName() + " bind " + getBindAddress() + ", export " + getLocalAddress());
        }
    } catch (Throwable t) {
        throw new RemotingException(url.toInetSocketAddress(), null, "Failed to bind " + getClass().getSimpleName()+ " on " + getLocalAddress() + ", cause: " + t.getMessage(), t);
    }
    //fixme replace this with better method,初始化线程池
    DataStore dataStore = ExtensionLoader.getExtensionLoader(DataStore.class).getDefaultExtension();
    executor = (ExecutorService) dataStore.get(Constants.EXECUTOR_SERVICE_COMPONENT_KEY, Integer.toString(url.getPort()));
}

NettyServer的doOpen方法:

// 绑定ip与端口
@Override
protected void doOpen() throws Throwable {
    NettyHelper.setNettyLoggerFactory();
    // boss、worker线程池
    ExecutorService boss = Executors.newCachedThreadPool(new NamedThreadFactory("NettyServerBoss", true));
    ExecutorService worker = Executors.newCachedThreadPool(new NamedThreadFactory("NettyServerWorker", true));
    ChannelFactory channelFactory = new NioServerSocketChannelFactory(boss, worker, getUrl().getPositiveParameter(Constants.IO_THREADS_KEY, Constants.DEFAULT_IO_THREADS));
    bootstrap = new ServerBootstrap(channelFactory);

    final NettyHandler nettyHandler = new NettyHandler(getUrl(), this);
    channels = nettyHandler.getChannels();
    bootstrap.setOption("child.tcpNoDelay", true);
    bootstrap.setPipelineFactory(new ChannelPipelineFactory() {
        @Override
        public ChannelPipeline getPipeline() {
            NettyCodecAdapter adapter = new NettyCodecAdapter(getCodec(), getUrl(), NettyServer.this);
            ChannelPipeline pipeline = Channels.pipeline();
            pipeline.addLast("decoder", adapter.getDecoder());
            pipeline.addLast("encoder", adapter.getEncoder());
            pipeline.addLast("handler", nettyHandler);
            return pipeline;
        }
    });
    // bind,最终调用netty的bootstrap.bind方法建立channel通道
    channel = bootstrap.bind(getBindAddress());
}

到这里,createServer的动作就完成了,其本质是借助netty创建服务端socket连接,并绑定ip和端口。再来看,若serverMap缓存中server已存在,则执行reset方法。核心逻辑是,若当前心跳值与url心跳值不一致或者当前空闲超时时间与url的空闲超时时间不一致,取消关闭任务(CloseTimerTask)并重启空闲连接检查任务(借助前面提到的HashedWheelTimer定时器实现)。代码如下:

@Override
public void reset(URL url) {
    server.reset(url);
    try {
        int currHeartbeat = UrlUtils.getHeartbeat(getUrl());
        int currIdleTimeout = UrlUtils.getIdleTimeout(getUrl());
        int heartbeat = UrlUtils.getHeartbeat(url);
        int idleTimeout = UrlUtils.getIdleTimeout(url);
        // 当前心跳值与url心跳值不一致或者当前空闲超时时间与url的空闲超时时间不一致,取消关闭任务并重启空闲检查任务。
        if (currHeartbeat != heartbeat || currIdleTimeout != idleTimeout) {
            cancelCloseTask();
            startIdleCheckTask(url);
        }
    } catch (Throwable t) {
        logger.error(t.getMessage(), t);
    }
}

2.2 refer(服务引用)

DubboProtocol中服务引用过程比较简单,大致分为连接Server、创建client数组,初始化Invoker,缓存至Invoker列表等几个步骤,最后后返回创建的Invoker。

@Override
public  Invoker refer(Class serviceType, URL url) throws RpcException {
  // 无有效逻辑,可以忽略  
  optimizeSerialization(url);
    //初始化Invoker,重点关注getClients
    DubboInvoker invoker = new DubboInvoker(serviceType, url, getClients(url), invokers);
  // 加入invokers列表缓存
  invokers.add(invoker);
  return invoker;
}

创建Client数据的过程比较有意思,首先,dubbo默认是不共享service连接的,即一个service一个连接,我们按照共享和不共享连接两种情况分析:

1、共享连接

        a、当用户没有自定义连接数时,dubbo会默认共享连接,且连接数默认为1;

        b、构建sharedClient列表,此时会优先从缓存map中取,若缓存的sharedClient列表内client均可用则直接返回;若缓存结果为空,则初始化sharedClient列表并缓存到map,返回sharedClient列表结果;若缓存结果非空,即缓存的sharedClient列表内部分client不可用的情况,则会新建client替换列表中不可用的client。注意:这里创建的是ReferenceCountExchangeClient实例

        c、将sharedClient列表内client复制到client数组,作为最终结果返回。

 2、不共享连接

      直接执行client数组的初始化(*initClient方法*)

核心代码如下:

private ExchangeClient[] getClients(URL url) {
    // 默认不共享连接
    boolean useShareConnect = false;
    int connections = url.getParameter(Constants.CONNECTIONS_KEY, 0);
    List shareClients = null;
    // 若用户没有自定义连接数,则dubbo认为共享连接,否则每个服务独享一个连接
    if (connections == 0) {
        useShareConnect = true;
        /**
         * The xml configuration should have a higher priority than properties.
         */
        String shareConnectionsStr = url.getParameter(Constants.SHARE_CONNECTIONS_KEY, (String) null);
        // 共享连接数默认是1
        connections = Integer.parseInt(StringUtils.isBlank(shareConnectionsStr) ? ConfigUtils.getProperty(Constants.SHARE_CONNECTIONS_KEY,Constants.DEFAULT_SHARE_CONNECTIONS) : shareConnectionsStr);
        // 构建共享client列表
        shareClients = getSharedClient(url, connections);
    }
    // clients.size >= 1
    ExchangeClient[] clients = new ExchangeClient[connections];
    for (int i = 0; i < clients.length; i++) {
        if (useShareConnect) {
            clients[i] = shareClients.get(i);
        } else {
            clients[i] = initClient(url);
        }
    }
    return clients;
}

构建共享client列表的逻辑在getSharedClient方法,篇幅原因,这里略去。在构建client列表过程中,都会涉及client的初始化,核心逻辑在initClient。initClient方法代码如下,其中属性值配置部分略过(属性值配置,同样默认采用NettyTransporter,默认心跳间隔60s,编解码默认采用DubboCodec):

private ExchangeClient initClient(URL url) {
        // 属性值配置及参数校验省略
    ExchangeClient client;
    try {
        // 延迟连接,即延迟到真正使用client时才会进行初始化
        if (url.getParameter(Constants.LAZY_CONNECT_KEY, false)) {
            client = new LazyConnectExchangeClient(url, requestHandler);
        } else {
            // 重点关注,借助org.apache.dubbo.remoting.exchange.Exchanger的connect方法,创建client
            client = Exchangers.connect(url, requestHandler);
        }
    } catch (RemotingException e) {
     throw new RpcException("Fail to create remoting client for service(" + url + "): " + e.getMessage(), e);
    }
  return client;
}

与2.1中export方法类似,初始化client方法借助Exchanger接口connect方法实现,同样以HeadExchanger为例,

@Override
public ExchangeClient connect(URL url, ExchangeHandler handler) throws RemotingException {
    return new HeaderExchangeClient(Transporters.connect(url, new DecodeHandler(new HeaderExchangeHandler(handler))), true);
}

可以看到,connect方法通过Transporter的connect方法实现,以NettyTransporter为例,直接实例化NettyClient并返回

@Override
public Client connect(URL url, ChannelHandler listener) throws RemotingException {
    return new NettyClient(url, listener);
}

NettyClient实例化过程与NettyServer类似,也是借助抽象基类(AbstractClient)构造方法完成。抽象基类AbstractClient定义了doOpen、doConnect、doClose、doDisconnect等模板方法,留给具体子类实现。在其构造方法中,除了部分属性初始化逻辑之外,会依次执行doOpen、connect。

// 篇幅原因,方法内异常处理、日志打印均省略
public AbstractClient(URL url, ChannelHandler handler) throws RemotingException {
    super(url, handler);
        // 是否需要重连,dubbo默认无需重连
    needReconnect = url.getParameter(Constants.SEND_RECONNECT_KEY, false);
    // 打开Channel
    doOpen();
    // 执行连接 connect.逻辑比较简单,内部实际还是执行具体子类的doConnect方法
    connect();
      // 初始化消费者端线程池,然后移除缓存中线程池
    executor = (ExecutorService) ExtensionLoader.getExtensionLoader(DataStore.class)
            .getDefaultExtension().get(Constants.CONSUMER_SIDE, Integer.toString(url.getPort()));
    ExtensionLoader.getExtensionLoader(DataStore.class)
            .getDefaultExtension().remove(Constants.CONSUMER_SIDE, Integer.toString(url.getPort()));
}

以NettyClient为例,来看doOpen与doConnect方法内部究竟做了什么:

@Override
protected void doOpen() throws Throwable {
        // 日志配置
    NettyHelper.setNettyLoggerFactory();
    // 初始化bootStrap
    bootstrap = new ClientBootstrap(channelFactory);
    // config,完善bootStrap参数配置
    // @see org.jboss.netty.channel.socket.SocketChannelConfig
    bootstrap.setOption("keepAlive", true);
    bootstrap.setOption("tcpNoDelay", true);
    bootstrap.setOption("connectTimeoutMillis", getConnectTimeout());
    final NettyHandler nettyHandler = new NettyHandler(getUrl(), this);
    bootstrap.setPipelineFactory(new ChannelPipelineFactory() {
        @Override
        public ChannelPipeline getPipeline() {
            NettyCodecAdapter adapter = new NettyCodecAdapter(getCodec(), getUrl(), NettyClient.this);
            ChannelPipeline pipeline = Channels.pipeline();
            pipeline.addLast("decoder", adapter.getDecoder());
            pipeline.addLast("encoder", adapter.getEncoder());
            pipeline.addLast("handler", nettyHandler);
            return pipeline;
        }
    });
}

NettyClient的doOpen方法,仅仅是对bootStrap做了初始化和部分属性的完善,再来看doConnect方法:

// 省略异常处理与日志打印
@Override
protected void doConnect() throws Throwable {
    long start = System.currentTimeMillis();
    // 通过netty建立连接,拿到NettyChannel
    ChannelFuture future = bootstrap.connect(getConnectAddress());
    try {
        // 非中断标志
        boolean ret = future.awaitUninterruptibly(getConnectTimeout(), TimeUnit.MILLISECONDS);
        if (ret && future.isSuccess()) {
            Channel newChannel = future.getChannel();
            newChannel.setInterestOps(Channel.OP_READ_WRITE);
            try {
                // Close old channel,关闭老的channel
                Channel oldChannel = NettyClient.this.channel; // copy reference
                if (oldChannel != null) {
                    try {
                        oldChannel.close();
                    } finally {
                        // 老的channel从缓存中删除
                        NettyChannel.removeChannelIfDisconnected(oldChannel);
                    }
                }
             } finally {
                // 若当前通道处于关闭状态,那么新建的通道也保持关闭状态
                if (NettyClient.this.isClosed()) {
                    try {
                        newChannel.close();
                    } finally {
                        NettyClient.this.channel = null;
                        NettyChannel.removeChannelIfDisconnected(newChannel);
                    }
                } else {
                    // 原本通道处于开启状态,那么当前通道直接替换为新建的channel
                    NettyClient.this.channel = newChannel;
                }
            }
       } else if (future.getCause() != null) {
        // 异常包装后抛出
    } finally {
        // 连接失败,则取消future
        if (!isConnected()) {
           future.cancel();
         }
    }
}

doConnect借助doOpen初始化后的bootStrap创建新的netty通道。有个细节需要注意,每次连接都会新建Channel,而且新建的Channel要与连接前的channel状态保持一致。到这里为止,采用HeaderExchangerClient、NettyTransporter的DubboProtocol的服务引用流程结束。

2.3 destroy(协议销毁)

destroy主要用作资源的回收,比如server、client的关闭等,直接来看代码:

public void destroy() {
    // 关闭serverMap缓存内所有server
    for (String key : new ArrayList<>(serverMap.keySet())) {
        ExchangeServer server = serverMap.remove(key);
        if (server == null) {
            continue;
        }
        try {
            if (logger.isInfoEnabled()) {
                logger.info("Close dubbo server: " + server.getLocalAddress());
            }
                    server.close(ConfigurationUtils.getServerShutdownTimeout());
        } catch (Throwable t) {
            logger.warn(t.getMessage(), t);
        }
    }
        
    // 关闭clientMap缓存内所有client
    for (String key : new ArrayList<>(referenceClientMap.keySet())) {
        List clients = referenceClientMap.remove(key);
                if (CollectionUtils.isEmpty(clients)) {
            continue;
        }
        for (ReferenceCountExchangeClient client : clients) {
            closeReferenceCountExchangeClient(client);
        }
    }
        // stubService方法缓存清空
    stubServiceMethodsMap.clear();
    // 父类destroy方法,参考AbstractProtocol的destroy方法
    super.destroy();
}

3、InjvmProtocol

InjvmProtocol即Protocol本地协议实现,仅支持服务本地暴露和引用,默认端口0。InjvmProtocol对export、refer非常简单

// export方法,新建InjvmExporter,放入父类的exporterMap
@Override
public  Exporter export(Invoker invoker) throws RpcException {
    return new InjvmExporter(invoker, invoker.getUrl().getServiceKey(), exporterMap);
}
// 顺便来看InjvmExporter的构造方法
InjvmExporter(Invoker invoker, String key, Map> exporterMap) {
        super(invoker);
        this.key = key;
        this.exporterMap = exporterMap;
        exporterMap.put(key, this);
}
// refer方法,新建InjvmInvoker,直接返回
@Override
public  Invoker refer(Class serviceType, URL url) throws RpcException {
    return new InjvmInvoker(serviceType, url, url.getServiceKey(), exporterMap);
}

除此之外,有两个地方需要注意,InjvmProtocol对外提供单例实现,线程安全由ExtensionLoader.getExtesion保证(内部DCL锁保证线程安全)代码如下:

private static InjvmProtocol INSTANCE;
public InjvmProtocol() {
        INSTANCE = this;
}

public static InjvmProtocol getInjvmProtocol() {
    if (INSTANCE == null) {
        //注意,这里如果直接new InjvmProtocol则是非线程安全的单例实现;
        ExtensionLoader.getExtensionLoader(Protocol.class).getExtension(InjvmProtocol.NAME); // load
    }
    return INSTANCE;
}

另一个需要注意的地方是对本地引用的判断,如果url中的参数scope=local,或者参数injvm=true会被判断为本地引用;另外,若本地有服务暴露,则也会通过本地引用,具体代码如下:

public boolean isInjvmRefer(URL url) {
    String scope = url.getParameter(Constants.SCOPE_KEY);
    // 对于本地引用来说,scope = local 与 injvm = true是完全等价的
    if (Constants.SCOPE_LOCAL.equals(scope) || (url.getParameter(Constants.LOCAL_PROTOCOL, false))) {
        return true;
    } else if (Constants.SCOPE_REMOTE.equals(scope)) {
        // 远程引用
        return false;
    } else if (url.getParameter(Constants.GENERIC_KEY, false)) {
        // 通用invocation非本地引用
        return false;
    } else if (getExporter(exporterMap, url) != null) {
        // 默认情况下,如果有本地服务暴露,则通过本地引用
        return true;
    } else {
        return false;
    }
}

4、RedisProtocol

RedisProtocol仅支持refer操作,协议默认端口6379,内部借助jedis实现,将url中get、set、delete参数值(默认值分别为get、set、delete)分别与会话域(Invocation)中的的方法名、参数(有且仅有一个参数)匹配,若匹配成功,则返回一个Invoker实现(invoke方法内部会执行redis的get(或者set、delete方法,并返回操作结果)。核心代码如下:

// 其他参数设置代码省略
final String get = url.getParameter("get", "get");
final String set = url.getParameter("set", Map.class.equals(type) ? "put" : "set");
final String delete = url.getParameter("delete", Map.class.equals(type) ? "remove" : "delete");
return new AbstractInvoker(type, url) {
    @Override
    protected Result doInvoke(Invocation invocation) throws Throwable {
        Jedis jedis = null;
        try {
            jedis = jedisPool.getResource();

            // get方法名取自url中的get参数,该方法有且只能有一个参数
            if (get.equals(invocation.getMethodName())) {
                if (invocation.getArguments().length != 1) {
                    throw new IllegalArgumentException("The redis get method arguments mismatch, must only one arguments. interface: " + type.getName() + ", method: " + invocation.getMethodName() + ", url: " + url);
                }
                // 执行redis get操作
                byte[] value = jedis.get(String.valueOf(invocation.getArguments()[0]).getBytes());
                if (value == null) {
                    return new RpcResult();
                }
                ObjectInput oin = getSerialization(url).deserialize(url, new ByteArrayInputStream(value));
                return new RpcResult(oin.readObject());
            } else if (set.equals(invocation.getMethodName())) {
                if (invocation.getArguments().length != 2) {
                    throw new IllegalArgumentException("The redis set method arguments mismatch, must be two arguments. interface: " + type.getName() + ", method: " + invocation.getMethodName() + ", url: " + url);
                }
                byte[] key = String.valueOf(invocation.getArguments()[0]).getBytes();
                ByteArrayOutputStream output = new ByteArrayOutputStream();
                ObjectOutput value = getSerialization(url).serialize(url, output);
                value.writeObject(invocation.getArguments()[1]);
                // 执行redis的set操作
                jedis.set(key, output.toByteArray());
                if (expiry > 1000) {
                    jedis.expire(key, expiry / 1000);
                }
                return new RpcResult();
            } else if (delete.equals(invocation.getMethodName())) {
                if (invocation.getArguments().length != 1) {
                    throw new IllegalArgumentException("The redis delete method arguments mismatch, must only one arguments. interface: " + type.getName() + ", method: " + invocation.getMethodName() + ", url: " + url);
                }
                // 执行redis的delete操作
                jedis.del(String.valueOf(invocation.getArguments()[0]).getBytes());
                return new RpcResult();
            } else {
                throw new UnsupportedOperationException("Unsupported method " + invocation.getMethodName() + " in redis service.");
            }
        } catch (Throwable t) {
            RpcException re = new RpcException("Failed to invoke redis service method. interface: " + type.getName() + ", method: " + invocation.getMethodName() + ", url: " + url + ", cause: " + t.getMessage(), t);
            if (t instanceof TimeoutException || t instanceof SocketTimeoutException) {
                re.setCode(RpcException.TIMEOUT_EXCEPTION);
            } else if (t instanceof JedisConnectionException || t instanceof IOException) {
                re.setCode(RpcException.NETWORK_EXCEPTION);
            } else if (t instanceof JedisDataException) {
                re.setCode(RpcException.SERIALIZATION_EXCEPTION);
            }
            throw re;
        } finally {
            if (jedis != null) {
                try {
                    // 关闭redis
                    jedis.close();
                } catch (Throwable t) {
                    logger.warn("returnResource error: " + t.getMessage(), t);
                }
            }
        }
    }

    @Override
    public void destroy() {
        super.destroy();
        try {
            // 销毁jedis连接池
            jedisPool.destroy();
        } catch (Throwable e) {
            logger.warn(e.getMessage(), e);
        }
    }
};

5、MemcachedProtocol

MemcachedProtocol同样仅支持refer操作,默认端口 11211,内部借助google的工具xmemcached实现(有兴趣的同学可以了解xmemcached的使用,jar包版本如下),逻辑与RedisProcotol的refer方法类似,url中get、set、delete参数值(默认值分别为get、set、delete)分别与会话域中的的方法名、参数(有且仅有一个参数)匹配,若匹配上返回一个Invoker实现(Invoke方法会执行memcache的get(或者set、delete方法),并返回操作结果)。


  com.googlecode.xmemcached
  xmemcached
  1.3.6

核心代码如下:

// 其他参数设置类代码省略
final int expiry = url.getParameter("expiry", 0);
final String get = url.getParameter("get", "get");
final String set = url.getParameter("set", Map.class.equals(type) ? "put" : "set");
final String delete = url.getParameter("delete", Map.class.equals(type) ? "remove" : "delete");
return new AbstractInvoker(type, url) {
    @Override
    protected Result doInvoke(Invocation invocation) throws Throwable {
        try {
            if (get.equals(invocation.getMethodName())) {
                if (invocation.getArguments().length != 1) {
                    throw new IllegalArgumentException("The memcached get method arguments mismatch, must only one arguments. interface: " + type.getName() + ", method: " + invocation.getMethodName() + ", url: " + url);
                }
                // 执行memecache的get操作
                return new RpcResult(memcachedClient.get(String.valueOf(invocation.getArguments()[0])));
            } else if (set.equals(invocation.getMethodName())) {
                // 参数长度非法
                if (invocation.getArguments().length != 2) {
                    throw new IllegalArgumentException("The memcached set method arguments mismatch, must be two arguments. interface: " + type.getName() + ", method: " + invocation.getMethodName() + ", url: " + url);
                }
                // 执行memcache的set操作 
                memcachedClient.set(String.valueOf(invocation.getArguments()[0]), expiry, invocation.getArguments()[1]);
                return new RpcResult();
            } else if (delete.equals(invocation.getMethodName())) {
                if (invocation.getArguments().length != 1) {
                    throw new IllegalArgumentException("The memcached delete method arguments mismatch, must only one arguments. interface: " + type.getName() + ", method: " + invocation.getMethodName() + ", url: " + url);
                }
                // 执行memcache的delete操作
                memcachedClient.delete(String.valueOf(invocation.getArguments()[0]));
                return new RpcResult();
            } else {
                throw new UnsupportedOperationException("Unsupported method " + invocation.getMethodName() + " in memcached service.");
            }
        } catch (Throwable t) {
            RpcException re = new RpcException("Failed to invoke memcached service method. interface: " + type.getName() + ", method: " + invocation.getMethodName() + ", url: " + url + ", cause: " + t.getMessage(), t);
            if (t instanceof TimeoutException || t instanceof SocketTimeoutException) {
                re.setCode(RpcException.TIMEOUT_EXCEPTION);
            } else if (t instanceof MemcachedException || t instanceof IOException) {
                re.setCode(RpcException.NETWORK_EXCEPTION);
            }
            throw re;
        }
    }

    @Override
    public void destroy() {
        super.destroy();
        try {
            // memcache关闭操作
            memcachedClient.shutdown();
        } catch (Throwable e) {
            logger.warn(e.getMessage(), e);
        }
    }
};

小结

上面介绍的几种procotol直接继承基类AbstractProtocol,都比较容易理解。其中DubboProtocol比较复杂,完全由dubbo自己实现,仅在tansport层(网络传输层)依赖了netty的网络通信能力,相关的分析参考DubboProtocol章节。除了上面介绍的几种Protocol,还有一类protocol具备代理能力,即下面章节将要介绍的AbstractProxyProtocol。

6、AbstractProxyProtocol

6.1、基础方法实现

AbstractProxyProtocol在AbstractProtocol基类基础上引入ProxyFactory,同样实现了export、refer方法(内部主要借助定义的doExport、doRefer模板方法实现,模板方法由具体子类实现)。AbstractProxyProtocol的子类实现有HessianProtocol、HttpProtocol、RestProtocol、RmiProtocol、WebServiceProtocol。先来看export、refer方法:

public  Exporter export(final Invoker invoker) throws RpcException {
    final String uri = serviceKey(invoker.getUrl());
    // 优先从缓存取
    Exporter exporter = (Exporter) exporterMap.get(uri);
    if (exporter != null) {
        // exporter的url信息不一致时,重新执行暴露逻辑;url信息一致则直接返回
        if (Objects.equals(exporter.getInvoker().getUrl(), invoker.getUrl())) {
            return exporter;
        }
    }
    // doExport方法由具体子类实现,这里会借助proxyFacotry生成代理实例
    final Runnable runnable = doExport(proxyFactory.getProxy(invoker, true), invoker.getInterface(), invoker.getUrl());
    // 匿名类实现,初始化exporter的Invoker,重写unExport方法
    exporter = new AbstractExporter(invoker) {
        @Override
        public void unexport() {
            super.unexport();
            exporterMap.remove(uri);
            if (runnable != null) {
                try {
                    runnable.run();
                } catch (Throwable t) {
                    logger.warn(t.getMessage(), t);
                }
            }
        }
    };
    // exporter放入exporterMap缓存
    exporterMap.put(uri, exporter);
    return exporter;
}

再来看refer方法:

@Override
public  Invoker refer(final Class type, final URL url) throws RpcException {
    // 借助proxyFactory生成Invoker代理实例,内部调用doRefer方法
    final Invoker target = proxyFactory.getInvoker(doRefer(type, url), type, url);
    Invoker invoker = new AbstractInvoker(type, url) {
        @Override
        protected Result doInvoke(Invocation invocation) throws Throwable {
            try {
                // 具体代理Invoker对象执行invoke逻辑
                Result result = target.invoke(invocation);
                // 有异常,则包装为rpcException之后,重新抛出
                Throwable e = result.getException();
                if (e != null) {
                    for (Class rpcException : rpcExceptions) {
                        if (rpcException.isAssignableFrom(e.getClass())) {
                            throw getRpcException(type, url, invocation, e);
                        }
                    }
                }
                return result;
            } catch (RpcException e) {
                if (e.getCode() == RpcException.UNKNOWN_EXCEPTION) {
                    e.setCode(getErrorCode(e.getCause()));
                }
                throw e;
            } catch (Throwable e) {
                throw getRpcException(type, url, invocation, e);
            }
        }
    };
    // 保存到invokers缓存列表
    invokers.add(invoker);
    return invoker;
}

逻辑比较简单,内部核心的doExport、doRefer方法由具体子类实现(ProxyFactory部分后面会开篇进行分析),来看这两个模板方法的定义:

// 模板方法 doExport
protected abstract  Runnable doExport(T impl, Class type, URL url) throws RpcException;
// 模板方法 doRefer
protected abstract  T doRefer(Class type, URL url) throws RpcException;

6.2、核心辅助接口

了解完AbstractProxyProtocol内部实现,介绍具体子类实现之前,先来看几个辅助接口:HttpServer、HttpBinder、HttpHandler。HttpBinder用于绑定HttpServer的具体执行动作(由HttpHandler实现),HttpServer则是所有HttpServer的抽象。其中,HttpBinder支持SPI,默认SPI实现是JettyHttpBinder。

6.2.1、HttpServer

HttpServer接口继承Resetable,即支持reset功能。除此之外,提供基本的server关闭等功能。子类通过继承抽象基类AbstractHttpServer,实现类包括JettyHttpServer、ServletHttpServer、TomcatHttpServer,从名字可以看出,主要是借助Jetty、Tomcat、Servlet容器为dubbo的服务暴露提供支持。下面按照顺序依次进行分析,核心逻辑都在构造方法,先来看JettyHttpServer。

6.2.1.1 JettyHttpServer

JettyHttpServer的构造方法主要完成JettyServer的初始化和启动:1、创建并初始化JettyServer;2、启动JettyServer:

public JettyHttpServer(URL url, final HttpHandler handler) {
    super(url, handler);
    this.url = url;
    // TODO we should leave this setting to slf4j
    // we must disable the debug logging for production use
    Log.setLog(new StdErrLog());
    Log.getLog().setDebugEnabled(false);
    // handler交给dispatcherServlet统一托管
    DispatcherServlet.addHttpHandler(url.getParameter(Constants.BIND_PORT_KEY, url.getPort()), handler);
    // 默认线程数200,使用jetty的队列线程池
    int threads = url.getParameter(Constants.THREADS_KEY, Constants.DEFAULT_THREADS);
    QueuedThreadPool threadPool = new QueuedThreadPool();
    threadPool.setDaemon(true);
    threadPool.setMaxThreads(threads);
    threadPool.setMinThreads(threads);

    // 创建并初始化nettyServer,指定JettyServer线程池大小、connector、以及Handler
    server = new Server(threadPool);
    ServerConnector connector = new ServerConnector(server);
    String bindIp = url.getParameter(Constants.BIND_IP_KEY, url.getHost());
    if (!url.isAnyHost() && NetUtils.isValidLocalHost(bindIp)) {
        connector.setHost(bindIp);
    }
    connector.setPort(url.getParameter(Constants.BIND_PORT_KEY, url.getPort()));
    server.addConnector(connector);

    // DispatcherServlet 托管给ServletHandler
    ServletHandler servletHandler = new ServletHandler();
    ServletHolder servletHolder = servletHandler.addServletWithMapping(DispatcherServlet.class, "/*");
    servletHolder.setInitOrder(2);

    // 设置JettyServer的handler 为ServletContextHandler
    ServletContextHandler context = new ServletContextHandler(server, "/", ServletContextHandler.SESSIONS);
    context.setServletHandler(servletHandler);
    ServletManager.getInstance().addServletContext(url.getParameter(Constants.BIND_PORT_KEY, url.getPort()), context.getServletContext());

    try {
        // 启动nettyServer
        server.start();
    } catch (Exception e) {
        throw new IllegalStateException("Failed to start jetty server on " + url.getParameter(Constants.BIND_IP_KEY) + ":" + url.getParameter(Constants.BIND_PORT_KEY) + ", cause: "
            + e.getMessage(), e);
    }
}

这块代码看起来比较好理解,但是有许多细节需要注意,比如自定义的Handler逻辑如何执行?什么时候执行?首先,代码注释中做了说明,自定义Handler会交给DispatcherServlet管理;然后,创建ServletHandler实例,调用ServertHandler.addServletWithMapping方法将DispatcherServlet交给Jetty的ServeltHandler;再接着,创建ServletContextHandler,并将该ServletHandler传递给该实例,最后将JettyServer(父类HandlerWrapper的属性)的_handler引用指向创建的ServletContextHandler实例,自定义的Handler就完全托管给JettyServer了,JettyServer的启动流程如下:

DispatcherServlet.addHttpHandler 
-> new QueueThreadPool()
-> 新建JettyServer实例,new Server() 
-> 同步执行ServletHandler.addServletWithMapping 
-> new ServletContextHandler()当前handler托管给JettyServer 
-> 启动JettyServer,JettyServer.start

下一个问题是handler的handle逻辑什么时候执行呢,可以肯定的一点是,handle一定是通过DispatcherServlet的service方法来执行,来看逻辑:

@Override
protected void service(HttpServletRequest request, HttpServletResponse response)
        throws ServletException, IOException {
    // 根据端口获取具体使用的handler
    HttpHandler handler = handlers.get(request.getLocalPort());
    if (handler == null) {// service not found.
        response.sendError(HttpServletResponse.SC_NOT_FOUND, "Service not found.");
    } else {
        // 具体的handle逻辑在这里执行
        handler.handle(request, response);
    }
}

这么简单吗?当然不是,下着来看,执行流程如下:

JettyServer初始化过程中,初始化线程池
new QueuedThreadPool() -> _runnable = new Runnable() 
-> jetty线程池异步调度执行 -> runJob()-> ChannelEndPoint._runFillalbe -> FillInterest.fillable()
-> AbstractConnection.ReadCallback.succeeded() -> HttpConnection.onFillable() 
-> HttpChannelOverHttp.handle()
-> Server.handle()(JettyServer构造过程中,会把传入的handler塞到Server中) 
-> 执行具体Server子类的Handler的handle方法

详细流程有兴趣的话可以参考Jetty的QueueThreadPool实现。

6.2.1.2 、TomcatHttpServer

与JettyHttpServer类似,TomcatHttpServer为dubbo提供web容器能力,核心逻辑在构造方法,同样包括两部分,Tomcat容器初始化和容器的启动,代码如下:

public TomcatHttpServer(URL url, final HttpHandler handler) {
        super(url, handler);

        this.url = url;
            // 同样的,自定义handler托管给DispatcherServlet
        DispatcherServlet.addHttpHandler(url.getPort(), handler);
        String baseDir = new File(System.getProperty("java.io.tmpdir")).getAbsolutePath();
            // tomcat属性配置,与server.xml中配置项等同
        tomcat = new Tomcat();
        tomcat.setBaseDir(baseDir);
        tomcat.setPort(url.getPort());
        tomcat.getConnector().setProperty("maxThreads",     String.valueOf(url.getParameter(Constants.THREADS_KEY, Constants.DEFAULT_THREADS)));
                tomcat.getConnector().setProperty(
                "maxConnections", String.valueOf(url.getParameter(Constants.ACCEPTS_KEY, -1)));
        tomcat.getConnector().setProperty("URIEncoding", "UTF-8");
        tomcat.getConnector().setProperty("connectionTimeout", "60000");
        tomcat.getConnector().setProperty("maxKeepAliveRequests", "-1");
        tomcat.getConnector().setProtocol("org.apache.coyote.http11.Http11NioProtocol");

            // DisptatcherServlet交给Tomcat的context管理
        Context context = tomcat.addContext("/", baseDir);
        Tomcat.addServlet(context, "dispatcher", new DispatcherServlet());
        context.addServletMapping("/*", "dispatcher");
        ServletManager.getInstance().addServletContext(url.getPort(), context.getServletContext());

        try {
            // 启动tomcat容器
            tomcat.start();
        } catch (LifecycleException e) {
            throw new IllegalStateException("Failed to start tomcat server at " + url.getAddress(), e);
        }
    }

同样的,我们来看自定义的handler的执行流程,DispatcherServlet入口就不再介绍了,与JettyHttpServer一样通过线程池异步执行,直接来看Tomcat的调度流程,有兴趣的同学可以自己研究下Tomcat的工作流程

线程池内工作线程队列,SocketProcessorBase.run() -> NioEndpoint$SocketProcessor.doRun() -> AbstractProtocol.process()
-> AbstractProcessorLight.process() -> Http11Processor.service()
-> CoyoteAdapter.service() -> StandardEngineValve.invoke()
-> ErrorReportValve.invoke() -> StandardHostValve.invoke()
-> AuthenticatorBase.invoke() -> StandardContextValve.invoke()
-> StandardWrapperValve.invoke() -> ApplicationFilterChain.doFilter()
-> ApplicationFilterChain.internalDoFilter() -> HttpServlet.service() -> DispatcherServlet.service()
-> 执行具体的handle逻辑
6.2.1.3 、ServletHttpServer

ServletHttpServer比较简单,直接使用HttpServlet作为web容器,代码也比较简单,不做过多解析。

public ServletHttpServer(URL url, HttpHandler handler) {
    super(url, handler);
    DispatcherServlet.addHttpHandler(url.getParameter(Constants.BIND_PORT_KEY, 8080), handler);
}

综上,HttpServer接口主要提供web容器,借助HttpBinder将dubbo服务暴露URL与web容器绑定,由web容器统一管理消费者请求。

6.2.2、HttpBinder

HttpBinder负责dubbo服务与web容器的绑定,接口支持SPI扩展,默认实现是JettyHttpBinder,即默认使用Jetty作为web容器。当然,可以通过URL参数指定容器类型,比如 &server=tomcat指定使用Tomcat。同时支持自定义Handler,用于web容器对绑定URL的处理。HttpBinder的逻辑非常简单,这里以JettyHttpBinder为例,代码如下:

@Override
public HttpServer bind(URL url, HttpHandler handler) {
    // 绑定URL与web容器,同时指定handler
    return new JettyHttpServer(url, handler);
}
6.2.3、HttpHandler

HttpHandler接口只有一个通用方法hanlde,在dubbo请求过程中,对请求做处理。

void handle(HttpServletRequest request, HttpServletResponse response) throws IOException, ServletException;

HttpHandler主要实现类有HessianProtocol.HessianHandler、HttpProtocol.InternalHandler、WebServiceProtocol.WebServiceHandler,分别位于对应的Protocol实现中,下面从HessianHandler开始,逐一解析。HessianHandler核心逻辑是借助HessianSkeleton完成单次rpc请求,代码如下:

private class HessianHandler implements HttpHandler {

    // 借助HessianSkeleton,实现rpc请求
    // 什么时候执行该handler?
    // 对于jetty来说,构建QueuedThreadPool时,会从自己的任务队列取出任务,调用自己的runJob方法,执行Runnable逻辑(实际执行的是ChannelEndPoint的_runFillable的run方法)
    // 大致流程: QueuedThreadPool -> QueuedThreadPool.runJob() -> ChannelEndPoint._runFillalbe -> FillInterest.fillable() -> AbstractConnection.ReadCallback.succeeded() -> HttpConnection.onFillable() ->  HttpChannelOverHttp.handle() -> Server.handle()(JettyServer构造过程中,会把传入的handler塞到Server中) -> 具体Server子类的Handler的handle方法
    @Override
    public void handle(HttpServletRequest request, HttpServletResponse response)
            throws IOException, ServletException {
        String uri = request.getRequestURI();
        HessianSkeleton skeleton = skeletonMap.get(uri);
        //仅支持post方法,貌似2.7.1以上版本支持其他类型请求
        if (!request.getMethod().equalsIgnoreCase("POST")) {
            response.setStatus(500);
        } else {
            RpcContext.getContext().setRemoteAddress(request.getRemoteAddr(), request.getRemotePort());
            Enumeration enumeration = request.getHeaderNames();
            while (enumeration.hasMoreElements()) {
                String key = enumeration.nextElement();
                if (key.startsWith(Constants.DEFAULT_EXCHANGER)) {
                   RpcContext.getContext().setAttachment(key.substring(Constants.DEFAULT_EXCHANGER.length()),
                            request.getHeader(key));
                }
            }

            try {
                // 调用Hesssian的invoke方法
                skeleton.invoke(request.getInputStream(), response.getOutputStream());
            } catch (Throwable e) {
                throw new ServletException(e);
            }
        }
    }
}

再来看WebServiceHandler,借助apache的cxf,使用ServletController 完成rpc请求过程,代码如下:

private class WebServiceHandler implements HttpHandler {
    private volatile ServletController servletController;
    @Override
    public void handle(HttpServletRequest request, HttpServletResponse response) throws IOException, ServletException {
        if (servletController == null) {
            HttpServlet httpServlet = DispatcherServlet.getInstance();
            if (httpServlet == null) {
                response.sendError(500, "No such DispatcherServlet instance.");
                return;
            }
            synchronized (this) {
                if (servletController == null) {
                    servletController = new ServletController(transportFactory.getRegistry(), httpServlet.getServletConfig(), httpServlet);
                }
            }
        }
        RpcContext.getContext().setRemoteAddress(request.getRemoteAddr(), request.getRemotePort());
        servletController.invoke(request, response);
    }
}

HttpProtocol对应的Handler实现即InternalHandler,利用spring的httpinvoker包中的HttpInvokerServiceExporter实现对请求的处理。代码如下

private class InternalHandler implements HttpHandler {
    @Override
    public void handle(HttpServletRequest request, HttpServletResponse response)
            throws IOException, ServletException {
        String uri = request.getRequestURI();
        HttpInvokerServiceExporter skeleton = skeletonMap.get(uri);
        // 同样仅支持post请求
        if (!request.getMethod().equalsIgnoreCase("POST")) {
            response.setStatus(500);
        } else {
            RpcContext.getContext().setRemoteAddress(request.getRemoteAddr(), request.getRemotePort());
            try {
                skeleton.handleRequest(request, response);
            } catch (Throwable e) {
                throw new ServletException(e);
            }
        }
    }
}

7、HessianProtocol

在介绍完AbstractProxyProtocol及相关辅助接口之后,我们来看HessianProtocol,重点关注doExport、doRefer方法,也是HessianProtocol服务暴露与服务引用的主要逻辑所在。Hessian的doExport核心逻辑,可以概括为两部分,创建HttpServer和创建Exporter并返回。创建HttpServer的过程借助HttpBinder实现,即调用HttpBinder的bind方法生成对应server,默认的实现是JettyHttpServer。生成Exporter则更为简单,只是创建了一个Runnable,用于实现Exporter的unExport方法,下面来看代码:

protected  Runnable doExport(T impl, Class type, URL url) throws RpcException {
    String addr = getAddr(url);
    // 先从server缓存取
    HttpServer server = serverMap.get(addr);
    if (server == null) {
        // 取不到则新建JettyHttpServer,并启动
        server = httpBinder.bind(url, new HessianHandler());
        serverMap.put(addr, server);
    }
    final String path = url.getAbsolutePath();
    final HessianSkeleton skeleton = new HessianSkeleton(impl, type);
    skeletonMap.put(path, skeleton);

    // 通用服务
    final String genericPath = path + "/" + Constants.GENERIC_KEY;
    skeletonMap.put(genericPath, new HessianSkeleton(impl, GenericService.class));

    return new Runnable() {
        @Override
        public void run() {
            skeletonMap.remove(path);
            skeletonMap.remove(genericPath);
        }
    };
}

再来看doRefer(),核心逻辑同样可以分为两部分,创建HessianProxyFactory、利用HessianProxyFactory生成接口的代理实现并返回

@Override
@SuppressWarnings("unchecked")
protected  T doRefer(Class serviceType, URL url) throws RpcException {
    String generic = url.getParameter(Constants.GENERIC_KEY);
    boolean isGeneric = ProtocolUtils.isGeneric(generic) || serviceType.equals(GenericService.class);
    if (isGeneric) {
        RpcContext.getContext().setAttachment(Constants.GENERIC_KEY, generic);
        url = url.setPath(url.getPath() + "/" + Constants.GENERIC_KEY);
    }

    HessianProxyFactory hessianProxyFactory = new HessianProxyFactory();
    boolean isHessian2Request = url.getParameter(Constants.HESSIAN2_REQUEST_KEY, Constants.DEFAULT_HESSIAN2_REQUEST);
    hessianProxyFactory.setHessian2Request(isHessian2Request);
    boolean isOverloadEnabled = url.getParameter(Constants.HESSIAN_OVERLOAD_METHOD_KEY, Constants.DEFAULT_HESSIAN_OVERLOAD_METHOD);
    hessianProxyFactory.setOverloadEnabled(isOverloadEnabled);
    String client = url.getParameter(Constants.CLIENT_KEY, Constants.DEFAULT_HTTP_CLIENT);
    // 客户端连接方式,httpclient,默认值jdk
    if ("httpclient".equals(client)) {
        HessianConnectionFactory factory = new HttpClientConnectionFactory();
        factory.setHessianProxyFactory(hessianProxyFactory);
        hessianProxyFactory.setConnectionFactory(factory);
    } else if (client != null && client.length() > 0 && !Constants.DEFAULT_HTTP_CLIENT.equals(client)) {
        // 非默认值,则直接抛异常
        throw new IllegalStateException("Unsupported http protocol client=\"" + client + "\"!");
    } else {
        // 默认采用Hessian连接
        HessianConnectionFactory factory = new DubboHessianURLConnectionFactory();
        factory.setHessianProxyFactory(hessianProxyFactory);
        hessianProxyFactory.setConnectionFactory(factory);
    }
    int timeout = url.getParameter(Constants.TIMEOUT_KEY, Constants.DEFAULT_TIMEOUT);
    hessianProxyFactory.setConnectTimeout(timeout);
    hessianProxyFactory.setReadTimeout(timeout);
    // 利用hessianProxyFactory生成引用服务实例,实际上是采用jdk的动态代理,生成serviceType实例
    // 前面部分完成HessianProxyFactory的各项初始化工作,执行create操作时,方法内部创建HessianProxy(实现JDK动态代理的InvocationHandler接口)实例,HessianProxy的_factory引用了当前的HessianProxyFactory对象,用于创建hessian连接时开启连接。整个hessian连接开启流程可以表示为:serviceType$Proxy.method -> HessianProxy.invoke ->  HessianProxy.sendRequest -> HessianProxyFactory.getConnectionFactory -> HessianConnectionFactory.open -> AbstractHessianOutput.call(执行被代理的方法,结果存放connection的outputStream) -> invoke方法调用结束
    // 这里有个疑问,每次调用结束之后HessianConnection都会直接关闭,下次请求过来再重新开启,如何保证性能?
    return (T) hessianProxyFactory.create(serviceType, url.setProtocol("http").toJavaURL(), Thread.currentThread().getContextClassLoader());
}

8、HttpProtocol

HttpProtocol借助Spring的httpinvoker以及HttpInvokerProxyFactoryBean实现服务的暴露和引用。服务暴露过程与HessianProtocol类似,包括两个步骤:创建Server、创建Exporter并返回,其中Server的创建过程同样借助前面提到的HttpBinder实现;服务引用则借助Spring的FactoryBean实现,即HttpInvokerProxyFactoryBean,最终返回的引用实例是FactoryBean.getObject。代码比较简单,这里就省略了。

9、RestProtocol

RestProtocol的doExport、doRefer方法核心逻辑与HttpProtocol大同小异,不同之处在于,1、dubbo为RestProtocol独立抽象出一个RestServer接口,也就是说在doExport过程中,创建的server是RestServer;2、服务引用过程借助于resteasy工具实现,最终采用ResteasyWebTarget生成代理服务实例(内部实际上还是JDK的动态代理实现)。这里我们详细介绍一下RestServer以及RestProtocol的服务引用的过程,先来看RestServer接口。

RestServer定义了REST服务的启动(start)、部署(deploy)、解除部署(undeploy)、停止(stop)等方法,其中,启动、部署、解除部署由基类BaseRestServer实现,stop方法则由具体子类实现。另外,BaseRestServer定义模板方法doStart、getDeployment,由子类具体实现。来看子类DubboHttpServer(名字取得是不是容易让人误解)、NettyServer。

9.1、DubboHttpServer

重点关注DubboHttpServer的doStrart方法,核心逻辑是创建HttpServer,初始化dispatcher。借助了resteasy的HttpServletDispatcher和ResteasyDeployment

@Override
protected void doStart(URL url) {
    // TODO jetty will by default enable keepAlive so the xml config has no effect now
    // 这里也借助了dubbo抽象的HttpBinder
    httpServer = httpBinder.bind(url, new RestHandler());

    ServletContext servletContext = ServletManager.getInstance().getServletContext(url.getPort());
    if (servletContext == null) {
        servletContext = ServletManager.getInstance().getServletContext(ServletManager.EXTERNAL_SERVER_PORT);
    }
    if (servletContext == null) {
        throw new RpcException("No servlet context found. If you are using server='servlet', " +
                "make sure that you've configured " + BootstrapListener.class.getName() + " in web.xml");
    }
    servletContext.setAttribute(ResteasyDeployment.class.getName(), deployment);
    try {
        // 初始化dispatcher
        dispatcher.init(new SimpleServletConfig(servletContext));
    } catch (ServletException e) {
        throw new RpcException(e);
    }
}

9.2、NettyServer

NettyServer比较简单,核心逻辑再doStart方法,负责初始化server(*resteasy的NettyJaxrsServer实例*),并启动:
@Override
protected void doStart(URL url) {
    String bindIp = url.getParameter(Constants.BIND_IP_KEY, url.getHost());
    if (!url.isAnyHost() && NetUtils.isValidLocalHost(bindIp)) {
        server.setHostname(bindIp);
    }
    // NettyJaxrsServer实例与ip地址、端口绑定,核心参数初始化
    server.setPort(url.getParameter(Constants.BIND_PORT_KEY, url.getPort()));
    Map channelOption = new HashMap();
    channelOption.put(ChannelOption.SO_KEEPALIVE, url.getParameter(Constants.KEEP_ALIVE_KEY, Constants.DEFAULT_KEEP_ALIVE));
    server.setChildChannelOptions(channelOption);
    server.setExecutorThreadCount(url.getParameter(Constants.THREADS_KEY, Constants.DEFAULT_THREADS));
    server.setIoWorkerCount(url.getParameter(Constants.IO_THREADS_KEY, Constants.DEFAULT_IO_THREADS));
    server.setMaxRequestSize(url.getParameter(Constants.PAYLOAD_KEY, Constants.DEFAULT_PAYLOAD));
    // 启动serfer  
    server.start();
}

来看doRefer过程,核心逻辑是初始化ResteasyClient,并将client加入list缓存,然后,构建ResteasyWebTarget,再通过ResteasyWebTarget的代理方法生成引用实例的代理,并返回,这里省略代码。

10、RmiProtocol

RmiProtocol的doExport逻辑,通过spring的RmiServiceExporter实现

@Override
protected  Runnable doExport(final T impl, Class type, URL url) throws RpcException {
    //初始化RmiServiceExporter,设置相关参数
    final RmiServiceExporter rmiServiceExporter = new RmiServiceExporter();
    rmiServiceExporter.setRegistryPort(url.getPort());
    rmiServiceExporter.setServiceName(url.getPath());
    rmiServiceExporter.setServiceInterface(type);
    rmiServiceExporter.setService(impl);
    try {
        // spring扩展逻辑
        rmiServiceExporter.afterPropertiesSet();
    } catch (RemoteException e) {
        throw new RpcException(e.getMessage(), e);
    }
    return new Runnable() {
        @Override
        public void run() {
            try {
                rmiServiceExporter.destroy();
            } catch (Throwable e) {
                logger.warn(e.getMessage(), e);
            }
        }
    };
}

doRefer方法则借助spring的RmiProxyFactoryBean实现,兼容2.7.0以下版本逻辑;方法最终返回RmiProxyFactoryBean所代理的bean实例。

protected  T doRefer(final Class serviceType, final URL url) throws RpcException {
    // 初始化RmiProxyFactoryBean
    final RmiProxyFactoryBean rmiProxyFactoryBean = new RmiProxyFactoryBean();
    if (isRelease270OrHigher(url.getParameter(Constants.RELEASE_KEY))) {
        rmiProxyFactoryBean.setRemoteInvocationFactory(RmiRemoteInvocation::new);
    } else if (isRelease263OrHigher(url.getParameter(Constants.DUBBO_VERSION_KEY))) {
        rmiProxyFactoryBean.setRemoteInvocationFactory(com.alibaba.dubbo.rpc.protocol.rmi.RmiRemoteInvocation::new);
    }
    rmiProxyFactoryBean.setServiceUrl(url.toIdentityString());
    rmiProxyFactoryBean.setServiceInterface(serviceType);
    rmiProxyFactoryBean.setCacheStub(true);
    rmiProxyFactoryBean.setLookupStubOnStartup(true);
    rmiProxyFactoryBean.setRefreshStubOnConnectFailure(true);
    rmiProxyFactoryBean.afterPropertiesSet();
    // 返回factoryBean代理对象实例
    return (T) rmiProxyFactoryBean.getObject();
}

11、WebServiceProtocol

WebServiceProtocol的doExport方法,借助apache的cxf(webService框架)工具包,通过ServerFactoryBean完成服务与实现的绑定:

@Override
protected  Runnable doExport(T impl, Class type, URL url) throws RpcException {
    String addr = getAddr(url);
    HttpServer httpServer = serverMap.get(addr);
    // 借助HttpBinder创建HttpServer
    if (httpServer == null) {
        httpServer = httpBinder.bind(url, new WebServiceHandler());
        serverMap.put(addr, httpServer);
    }
    final ServerFactoryBean serverFactoryBean = new ServerFactoryBean();
    serverFactoryBean.setAddress(url.getAbsolutePath());
    serverFactoryBean.setServiceClass(type);
    serverFactoryBean.setServiceBean(impl);
    serverFactoryBean.setBus(bus);
    serverFactoryBean.setDestinationFactory(transportFactory);
    serverFactoryBean.create();
    // Exporter匿名类,内部逻辑实现
    return new Runnable() {
        @Override
        public void run() {
            if(serverFactoryBean.getServer()!= null) {
                serverFactoryBean.getServer().destroy();
            }
            if(serverFactoryBean.getBus()!=null) {
                serverFactoryBean.getBus().shutdown(true);
            }
        }
    };
}

doRefer方法也是借助apache的cxf(webService框架),通过ClientProxyFactoryBean完成应用服务的实例化,并返回该实例。

@Override
@SuppressWarnings("unchecked")
protected  T doRefer(final Class serviceType, final URL url) throws RpcException {
    // 创建ClientProxyFactoryBean实例
    ClientProxyFactoryBean proxyFactoryBean = new ClientProxyFactoryBean();
    proxyFactoryBean.setAddress(url.setProtocol("http").toIdentityString());
    proxyFactoryBean.setServiceClass(serviceType);
    proxyFactoryBean.setBus(bus);
    // 动态代理创建服务引用实例
    T ref = (T) proxyFactoryBean.create();
    Client proxy = ClientProxy.getClient(ref);
    HTTPConduit conduit = (HTTPConduit) proxy.getConduit();
    HTTPClientPolicy policy = new HTTPClientPolicy();
    policy.setConnectionTimeout(url.getParameter(Constants.CONNECT_TIMEOUT_KEY, Constants.DEFAULT_CONNECT_TIMEOUT));
    policy.setReceiveTimeout(url.getParameter(Constants.TIMEOUT_KEY, Constants.DEFAULT_TIMEOUT));
    conduit.setClient(policy);
    return ref;
}

总结

Protocol在dubbo中的位置非常重要,本文按照接口、代理实现、直接实现三个部分对Protocol做了解析。总的来讲,Dubbo中Protocol的实现分为两种,1、代理实现,比如RegistryProtocol,并不直接实现Protocol,而是借助内部引用实例完成服务暴露、引用;2、直接实现,比如DubboProtocol,Server -> Exchanger -> Transporter -> 利用Netty建立socket连接,执行具体的服务暴露、引用;直接实现中还有一类,即借助web容器比如Jetty、tomcat、servlet或者三方框架如apache的cxf实现server的创建和启动,然后将dubbo服务URL、端口与server绑定,完成服务的暴露。

注:源码版本2.7.1,欢迎指正。

你可能感兴趣的:(dubbo之Protocol(协议))