3. Dubbo服务提供端请求处理流程

时序图

3. Dubbo服务提供端请求处理流程_第1张图片
服务提供端请求处理时序图.jpeg

从时序图上不难看出,服务提供端对请求的处理先通过处理器责任链一层一层处理,然后找到需要调用的服务实现类的代理Invoker进行调用,再将响应发送到调用方。

源码解析

  1. org.apache.dubbo.remoting.transport.AbstractPeer#received
    时序图步骤1
    NettyServer继承了AbstractPeer,AbstractPeer继承了ChannelHandler,所以NettyServer也是请求处理器.

调用NettyServer#handler处理请求

    @Override
    public void received(Channel ch, Object msg) throws RemotingException {
        if (closed) {
            return;
        }
        handler.received(ch, msg);
    }
  1. org.apache.dubbo.remoting.transport.MultiMessageHandler#received
    时序图步骤2
    MultiMessageHandler复合消息处理器,处理复合消息;
    MultiMessage(复合消息) 使用了迭代器模式;
    public void received(Channel channel, Object message) throws RemotingException {
        //如果是复合消息,需要拆分消息
        if (message instanceof MultiMessage) {
            MultiMessage list = (MultiMessage) message;
            //遍历,将MultiMessage中的message取出,一个一个处理
            for (Object obj : list) {
                handler.received(channel, obj);
            }
        } else {
            handler.received(channel, message);
        }
    }
  1. org.apache.dubbo.remoting.exchange.support.header.HeartbeatHandler#received
    时序图步骤3
    心跳处理器,处理心跳请求,返回心跳响应
    public void received(Channel channel, Object message) throws RemotingException {
        setReadTimestamp(channel);
      //是否是心跳请求
        if (isHeartbeatRequest(message)) {
            Request req = (Request) message;
            if (req.isTwoWay()) {
                //构建心跳响应并返回响应
                Response res = new Response(req.getId(), req.getVersion());
                res.setEvent(Response.HEARTBEAT_EVENT);
                channel.send(res);
                if (logger.isInfoEnabled()) {
                    int heartbeat = channel.getUrl().getParameter(Constants.HEARTBEAT_KEY, 0);
                    if (logger.isDebugEnabled()) {
                        logger.debug("Received heartbeat from remote channel " + channel.getRemoteAddress()
                                + ", cause: The channel has no data-transmission exceeds a heartbeat period"
                                + (heartbeat > 0 ? ": " + heartbeat + "ms" : ""));
                    }
                }
            }
        //直接返回,不需要调用其他处理器
            return;
        }
      //如果是心跳响应
        if (isHeartbeatResponse(message)) {
            if (logger.isDebugEnabled()) {
                logger.debug("Receive heartbeat response in thread " + Thread.currentThread().getName());
            }
        //直接返回,不需要调用其他处理器
            return;
        }
        //如果非心跳消息,交给处理链中后续处理器处理
        handler.received(channel, message);
    }
  1. org.apache.dubbo.remoting.transport.dispatcher.all.AllChannelHandler#received
    时序图步骤4

Dubbo默认的底层网络通信使用的是Netty,NettyServer使用两级线程池,EventLoopGroup(boss),主要用来接收客户端连接请求,并把完成TCP三次握手的连接分发给EventLoopGroup(worker)处理,根据请求的消息是被I/O线程处理还是被业务线程池处理,Dubbo提供了几种线程模型,AllDispatcher是默认线程模型,AllChannelHandler是对应的处理器.

AllDispatcher:将所有消息(请求,响应,心跳,连接事件,断开事件)都派发到业务线程池.

    public void received(Channel channel, Object message) throws RemotingException {
    //获取业务线程池   时序图步骤5
        ExecutorService cexecutor = getExecutorService();
        try {
          //将消息组装成status:receive的ChannelEventRunnable派发到业务线程池  时序图步骤6
            cexecutor.execute(new ChannelEventRunnable(channel, handler, ChannelState.RECEIVED, message));
        } catch (Throwable t) {
            //TODO A temporary solution to the problem that the exception information can not be sent to the opposite end after the thread pool is full. Need a refactoring
            //fix The thread pool is full, refuses to call, does not return, and causes the consumer to wait for time out
      //如果线程池已满,触发拒绝调用异常,返回异常响应
            if(message instanceof Request && t instanceof RejectedExecutionException){
                Request request = (Request)message;
                if(request.isTwoWay()){
                    String msg = "Server side(" + url.getIp() + "," + url.getPort() + ") threadpool is exhausted ,detail msg:" + t.getMessage();
                    Response response = new Response(request.getId(), request.getVersion());
                    response.setStatus(Response.SERVER_THREADPOOL_EXHAUSTED_ERROR);
                    response.setErrorMessage(msg);
                    channel.send(response);
                    return;
                }
            }
            throw new ExecutionException(message, channel, getClass() + " error when process received event .", t);
        }
    }
  1. org.apache.dubbo.remoting.transport.dispatcher.ChannelEventRunnable#run
    时序图步骤7

ChannelEventRunnable处理消息的线程


 public void run() {

        //请求
        if (state == ChannelState.RECEIVED) {
            try {
              //交给后续处理器
                handler.received(channel, message);
            } catch (Exception e) {
                logger.warn("ChannelEventRunnable handle " + state + " operation error, channel is " + channel
                        + ", message is " + message, e);
            }
        } else {
            switch (state) {
          //连接事件
            case CONNECTED:
                try {
                    handler.connected(channel);
                } catch (Exception e) {
                    logger.warn("ChannelEventRunnable handle " + state + " operation error, channel is " + channel, e);
                }
                break;
          //断开事件
            case DISCONNECTED:
                try {
                    handler.disconnected(channel);
                } catch (Exception e) {
                    logger.warn("ChannelEventRunnable handle " + state + " operation error, channel is " + channel, e);
                }
                break;
          //响应
            case SENT:
                try {
                    handler.sent(channel, message);
                } catch (Exception e) {
                    logger.warn("ChannelEventRunnable handle " + state + " operation error, channel is " + channel
                            + ", message is " + message, e);
                }
                break;
          //异常
            case CAUGHT:
                try {
                    handler.caught(channel, exception);
                } catch (Exception e) {
                    logger.warn("ChannelEventRunnable handle " + state + " operation error, channel is " + channel
                            + ", message is: " + message + ", exception is " + exception, e);
                }
                break;
            default:
                logger.warn("unknown state: " + state + ", message is " + message);
            }
        }

    }
  1. org.apache.dubbo.remoting.transport.DecodeHandler#received
    时序图步骤8

对消息进行解码,解码后交由后续处理器处理

    public void received(Channel channel, Object message) throws RemotingException {
        //如果消息实现了Decodeable,调用message 自身实现的decode函数
        if (message instanceof Decodeable) {
            decode(message);
        }
      //如果是请求,对org.apache.dubbo.remoting.exchange.Request#mData 解码
        if (message instanceof Request) {
            decode(((Request) message).getData());
        }
      //如果是响应,对org.apache.dubbo.remoting.exchange.Response#mResult解码
        if (message instanceof Response) {
            decode(((Response) message).getResult());
        }
      //讲解马后的消息交由后续处理器处理
        handler.received(channel, message);
    }
    private void decode(Object message) {
        if (message != null && message instanceof Decodeable) {
            try {
                ((Decodeable) message).decode();
                if (log.isDebugEnabled()) {
                    log.debug("Decode decodeable message " + message.getClass().getName());
                }
            } catch (Throwable e) {
                if (log.isWarnEnabled()) {
                    log.warn("Call Decodeable.decode failed: " + e.getMessage(), e);
                }
            } // ~ end of catch
        } // ~ end of if
    } 
  1. org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeHandler#received
    时序图步骤9

对消息进行分发处理

    public void received(Channel channel, Object message) throws RemotingException {
        channel.setAttribute(KEY_READ_TIMESTAMP, System.currentTimeMillis());
        final ExchangeChannel exchangeChannel = HeaderExchangeChannel.getOrAddChannel(channel);
        try {
            //请求消息
            if (message instanceof Request) {
                // handle request.
                Request request = (Request) message;
                if (request.isEvent()) {//事件处理
                    handlerEvent(channel, request);
                } else {
                    if (request.isTwoWay()) {//如果是需要返回响应的请求
                        handleRequest(exchangeChannel, request);
                    } else {//如果不需要返回响应,忽略处理器返回值
                        handler.received(exchangeChannel, request.getData());
                    }
                }
            } else if (message instanceof Response) {//响应消息
                handleResponse(channel, (Response) message);
            } else if (message instanceof String) {
                if (isClientSide(channel)) {//如果是客户端请求且消息是String 格式的,dubbo不支持string格式的客户端调用,记录异常日志,不进行后续处理
                    Exception e = new Exception("Dubbo client can not supported string message: " + message + " in channel: " + channel + ", url: " + channel.getUrl());
                    logger.error(e.getMessage(), e);
                } else {
                    //通过telnet调试的消息
                    String echo = handler.telnet(channel, (String) message);
                    if (echo != null && echo.length() > 0) {
                        channel.send(echo);
                    }
                }
            } else {
                handler.received(exchangeChannel, message);
            }
        } finally {
            HeaderExchangeChannel.removeChannelIfDisconnected(channel);
        }
    }

  1. org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeHandler#handleRequest
    时序图步骤10

处理消息,并返回响应

    void handleRequest(final ExchangeChannel channel, Request req) throws RemotingException {
        Response res = new Response(req.getId(), req.getVersion());
        //请求是否解码失败,如果解码失败,取出异常信息组合响应
        if (req.isBroken()) {
            Object data = req.getData();
            String msg;
            if (data == null) {
                msg = null;
            } else if (data instanceof Throwable) {
                msg = StringUtils.toString((Throwable) data);
            } else {
                msg = data.toString();
            }
            res.setErrorMessage("Fail to decode request due to: " + msg);
            res.setStatus(Response.BAD_REQUEST);

            channel.send(res);
            return;
        }
        // find handler by message class.
        Object msg = req.getData();
        try {
            // 调用org.apache.dubbo.rpc.protocol.dubbo.DubboProtocol#requestHandler进行消息处理,获取处理器返回值,这里是真正的服务实现类的调用了
            CompletableFuture future = handler.reply(channel, msg);
          //如果是服务实现类是同步处理,直接返回调用返回值
            if (future.isDone()) {
                res.setStatus(Response.OK);
                res.setResult(future.get());
          //时序图步骤16
                channel.send(res);
                return;
            }
          //如果服务实现类是异步处理,等待处理完成后回写响应
            future.whenComplete((result, t) -> {
                try {
                    if (t == null) {
                        res.setStatus(Response.OK);
                        res.setResult(result);
                    } else {
                        res.setStatus(Response.SERVICE_ERROR);
                        res.setErrorMessage(StringUtils.toString(t));
                    }
            //时序图步骤16
                    channel.send(res);
                } catch (RemotingException e) {
                    logger.warn("Send result to consumer failed, channel is " + channel + ", msg is " + e);
                } finally {
                    // HeaderExchangeChannel.removeChannelIfDisconnected(channel);
                }
            });
        } catch (Throwable e) {
            res.setStatus(Response.SERVICE_ERROR);
            res.setErrorMessage(StringUtils.toString(e));
            channel.send(res);
        }
    }
 
 
  1. org.apache.dubbo.rpc.protocol.dubbo.DubboProtocol#requestHandler#reply
    时序图步骤11
 public CompletableFuture reply(ExchangeChannel channel, Object message) throws RemotingException {

            if (!(message instanceof Invocation)) {
                throw new RemotingException(channel, "Unsupported request: "
                        + (message == null ? null : (message.getClass().getName() + ": " + message))
                        + ", channel: consumer: " + channel.getRemoteAddress() + " --> provider: " + channel.getLocalAddress());
            }

            Invocation inv = (Invocation) message;
          //利用inv找到对应的export,进而获取invoker 
            Invoker invoker = getInvoker(channel, inv);
            // need to consider backward-compatibility if it's a callback
            if (Boolean.TRUE.toString().equals(inv.getAttachments().get(IS_CALLBACK_SERVICE_INVOKE))) {
                String methodsStr = invoker.getUrl().getParameters().get("methods");
                boolean hasMethod = false;
                if (methodsStr == null || !methodsStr.contains(",")) {
                    hasMethod = inv.getMethodName().equals(methodsStr);
                } else {
                    String[] methods = methodsStr.split(",");
                    for (String method : methods) {
                        if (inv.getMethodName().equals(method)) {
                            hasMethod = true;
                            break;
                        }
                    }
                }
                if (!hasMethod) {
                    logger.warn(new IllegalStateException("The methodName " + inv.getMethodName()
                            + " not found in callback service interface ,invoke will be ignored."
                            + " please update the api interface. url is:"
                            + invoker.getUrl()) + " ,invocation is :" + inv);
                    return null;
                }
            }
          //获取上下文
            RpcContext rpcContext = RpcContext.getContext();
          //记录远程调用地址
            rpcContext.setRemoteAddress(channel.getRemoteAddress());
          //调用invoker的invoke函数,获取返回值
            Result result = invoker.invoke(inv);
            //如果是异步结果(AsyncRpcResult),返回org.apache.dubbo.rpc.AsyncRpcResult#resultFuture,当服务实现类处理结束后,将结果转成Object类型
            if (result instanceof AsyncRpcResult) {
                return ((AsyncRpcResult) result).getResultFuture().thenApply(r -> (Object) r);

            } else {//同步请求直接将结果作为result,返回已经完成的CompletableFuture
                return CompletableFuture.completedFuture(result);
            }
        }
 
 
  1. org.apache.dubbo.rpc.protocol.dubbo.DubboProtocol#getInvoker
    时序图步骤12

从inv中获取 path 、version、group,从channel中获取调用的端口,组装serviceKey ,找到缓存在exporterMap的DubboExporter,进而获取服务实现类代理对象Invoker

    Invoker getInvoker(Channel channel, Invocation inv) throws RemotingException {
        boolean isCallBackServiceInvoke = false;
        boolean isStubServiceInvoke = false;
        //获取服务端口
        int port = channel.getLocalAddress().getPort();
        //获取path
        String path = inv.getAttachments().get(Constants.PATH_KEY);

        // if it's callback service on client side
        isStubServiceInvoke = Boolean.TRUE.toString().equals(inv.getAttachments().get(Constants.STUB_EVENT_KEY));
        if (isStubServiceInvoke) {
            port = channel.getRemoteAddress().getPort();
        }

        //callback
        isCallBackServiceInvoke = isClientSide(channel) && !isStubServiceInvoke;
        if (isCallBackServiceInvoke) {
            path += "." + inv.getAttachments().get(Constants.CALLBACK_SERVICE_KEY);
            inv.getAttachments().put(IS_CALLBACK_SERVICE_INVOKE, Boolean.TRUE.toString());
        }
        //组装serviceKey ,grout/path:version:port 唯一的确定一个服务
        String serviceKey = serviceKey(port, path, inv.getAttachments().get(Constants.VERSION_KEY), inv.getAttachments().get(Constants.GROUP_KEY));
      //取出缓存在exporterMap的已发布服务
        DubboExporter exporter = (DubboExporter) exporterMap.get(serviceKey);

        if (exporter == null) {
            throw new RemotingException(channel, "Not found exported service: " + serviceKey + " in " + exporterMap.keySet() + ", may be version or group mismatch " +
                    ", channel: consumer: " + channel.getRemoteAddress() + " --> provider: " + channel.getLocalAddress() + ", message:" + inv);
        }

        return exporter.getInvoker();
    }

  1. org.apache.dubbo.rpc.proxy.AbstractProxyInvoker#invoke
    时序图步骤13,14

调用代理类,处理请求,根据返回类型组装相应的Result,如果是开启了异步上下文或者返回值为CompletableFuture类型,则返回AsyncRpcResult,
否则返回同步返回结果RpcResult

    public Result invoke(Invocation invocation) throws RpcException {
        RpcContext rpcContext = RpcContext.getContext();
        try {
            //调用子类实现,服务发布时通过调用org.apache.dubbo.rpc.proxy.javassist.JavassistProxyFactory#getInvoker构建了实现类代理对象
            Object obj = doInvoke(proxy, invocation.getMethodName(), invocation.getParameterTypes(), invocation.getArguments());
          //如果返回值是CompletableFuture类型
            if (RpcUtils.isReturnTypeFuture(invocation)) {
              //返回AsyncRpcResult,当实现类返回值:obj处理完后,会回调AsyncRpcResult#resultFuture,这里就不具体说了,后续会在Dubbo异步处理的章节详细分析
                return new AsyncRpcResult((CompletableFuture) obj);
          //如果开启了异步上下文
            } else if (rpcContext.isAsyncStarted()) { // ignore obj in case of RpcContext.startAsync()? always rely on user to write back.
              //取出异步上下文中的回调future,组装AsyncRpcResult
                return new AsyncRpcResult(((AsyncContextImpl)(rpcContext.getAsyncContext())).getInternalFuture());
            } else {//同步,直接返回RpcResult
                return new RpcResult(obj);
            }
        } catch (InvocationTargetException e) {
            // TODO async throw exception before async thread write back, should stop asyncContext
            if (rpcContext.isAsyncStarted() && !rpcContext.stopAsync()) {
                logger.error("Provider async started, but got an exception from the original method, cannot write the exception back to consumer because an async result may have returned the new thread.", e);
            }
            return new RpcResult(e.getTargetException());
        } catch (Throwable e) {
            throw new RpcException("Failed to invoke remote proxy method " + invocation.getMethodName() + " to " + getUrl() + ", cause: " + e.getMessage(), e);
        }
    }

 
 

总结

至此,Dubbo提供端请求处理流程就分析完了。总结一下:

  • NettyServer 收到客户端连接(CONNECTED),这里因为篇幅原因没有分析连接事件,但主体流程与请求流程是相似的.
  • 完成TCP 三次握手
  • 客户端发送 请求(RECEIVED)
  • 经由NettyServer 中构建的处理器责任链处理,复合消息拆分处理->心跳事件处理->根据线程模型分发请求->消息解码->..
  • 获取服务代理类
  • 调用服务代理类处理请求
  • 如果请求需要返回值,回写响应

你可能感兴趣的:(3. Dubbo服务提供端请求处理流程)