RocketMQ是如何通讯的?

前言

RocketMQ作为一个分布式消息中间件,通讯肯定是必不可少的。参与通讯的角色包括:NameSrv,Broker,Producer和Consumer,另外Broker Master和Slave结点之间也有通讯。RocketMQ的通讯基于Netty,在其基础上做了一层简单的封装。大致的通讯架构如下所示:
RocketMQ是如何通讯的?_第1张图片
理解的这个框架图里的组件,也就理解了通讯的整个过程。下面将以4.1.0版本进行介绍。

通讯载体RemotingCommand

不管是Producer发送消息、Consumer消费消息,还是注册、获取Topic路由信息,到底层RocketMQ通讯都是通过RemotingCommand这个类封装信息的,因此很有必要先了解一下这个类。

public class RemotingCommand {
    private static AtomicInteger requestId = new AtomicInteger(0);// 请求ID

    private static SerializeType serializeTypeConfigInThisServer = SerializeType.JSON; // 序列化类型

    private int code; // 请求代码
    private LanguageCode language = LanguageCode.JAVA; // 语言类型
    private int version = 0;
    private int opaque = requestId.getAndIncrement();
    private int flag = 0;
    private String remark;
    private HashMap<String, String> extFields;
    private transient CommandCustomHeader customHeader; // 自定义请求头

    private SerializeType serializeTypeCurrentRPC = serializeTypeConfigInThisServer; // 本次请求的序列化类型

    private transient byte[] body; // 请求体

    protected RemotingCommand() {
    }

为了突出重点,我省略了很多不相干的细节。RemotingCommand类变量有以下几个:

private static AtomicInteger requestId = new AtomicInteger(0);// 请求ID生成器,每次请求都会通过requestId生成一个递增的值并给到opaque,我们可以将opaque理解为真正的请求序列号。
private static SerializeType serializeTypeConfigInThisServer = SerializeType.JSON; // 序列化类型,RocketMQ支持JSON和ROCKETMQ两种序列化类型,默认JSON。只有Header部分可能会进行ROCKETMQ序列化。

RemotingCommand实例变量有以下几个:

    private int code; // 请求代码
    private LanguageCode language = LanguageCode.JAVA; // 语言类型
    private int version = 0; // 版本号
    private int opaque = requestId.getAndIncrement();
    private int flag = 0;
    private String remark;
    private HashMap extFields;
    private transient CommandCustomHeader customHeader; // 自定义请求头
    private transient byte[] body; // 请求体

code:请求代码,通讯接收端会根据这个code调用注册的processor处理该请求。code被定义在了RequestCode类里面,如下所示:
RocketMQ是如何通讯的?_第2张图片
language :语言类型,没什么好说的,默认JAVA。
version:mq版本,通讯接收方可能会根据版本做一些特殊操作,例如broker在发送消息时,如果mq版本大于v3.4.9,会从请求头里获取最大消费次数。

if (request.getVersion() >= MQVersion.Version.V3_4_9.ordinal()) {
            maxReconsumeTimes = requestHeader.getMaxReconsumeTimes();
        }

opaque:可以理解为请求序列号,可以唯一标识一个请求。在处理通讯返回时很有用处。
flag:标志位,通过位运算可以判断该请求是单向请求还是有返回的请求,还可以判断该RomotingCommand是请求还是返回。
remark:用来填充一些备注信息,例如异常信息。
extFields:存放扩展字段。
customHeader:自定义请求头,不会序列化。不同的请求有不同的请求头,例如创建topic请求的请求头如下所示:

public class CreateTopicRequestHeader implements CommandCustomHeader {
    @CFNotNull
    private String topic;
    @CFNotNull
    private String defaultTopic;
    @CFNotNull
    private Integer readQueueNums;
    @CFNotNull
    private Integer writeQueueNums;
    @CFNotNull
    private Integer perm;
    @CFNotNull
    private String topicFilterType;
    private Integer topicSysFlag;
    @CFNotNull
    private Boolean order = false;
    ......
    }

可以看到许多重要的信息都会放在请求头里。
body:请求内容,不会序列化。body一般存放请求内容,例如producer发送的消息,客户端与broker的心跳信息等。

RemotingClient与RemotingServer

broker和nameSrv分别通过RemotingClient和RemotingServer处理通讯的,两者皆是接口,均继承了RemotingService接口,如下所示:
RocketMQ是如何通讯的?_第3张图片
RemotingService接口非常简单,仅仅包含三个方法。其中registerRPCHook用于注册钩子方法,在请求发送前和收到返回后进行一些额外的预处理。

public interface RemotingService {
    void start();

    void shutdown();

    void registerRPCHook(RPCHook rpcHook);
}

RemotingClient和RemotingServer的定义如下:

public interface RemotingClient extends RemotingService {

    public void updateNameServerAddressList(final List<String> addrs);

    public List<String> getNameServerAddressList();

    public RemotingCommand invokeSync(final String addr, final RemotingCommand request,
        final long timeoutMillis) throws InterruptedException, RemotingConnectException,
        RemotingSendRequestException, RemotingTimeoutException;

    public void invokeAsync(final String addr, final RemotingCommand request, final long timeoutMillis,
        final InvokeCallback invokeCallback) throws InterruptedException, RemotingConnectException,
        RemotingTooMuchRequestException, RemotingTimeoutException, RemotingSendRequestException;

    public void invokeOneway(final String addr, final RemotingCommand request, final long timeoutMillis)
        throws InterruptedException, RemotingConnectException, RemotingTooMuchRequestException,
        RemotingTimeoutException, RemotingSendRequestException;

    public void registerProcessor(final int requestCode, final NettyRequestProcessor processor,
        final ExecutorService executor);

    public boolean isChannelWriteable(final String addr);
}

public interface RemotingServer extends RemotingService {

    void registerProcessor(final int requestCode, final NettyRequestProcessor processor,
        final ExecutorService executor);

    void registerDefaultProcessor(final NettyRequestProcessor processor, final ExecutorService executor);

    int localListenPort();

    Pair<NettyRequestProcessor, ExecutorService> getProcessorPair(final int requestCode);

    RemotingCommand invokeSync(final Channel channel, final RemotingCommand request,
        final long timeoutMillis) throws InterruptedException, RemotingSendRequestException,
        RemotingTimeoutException;

    void invokeAsync(final Channel channel, final RemotingCommand request, final long timeoutMillis,
        final InvokeCallback invokeCallback) throws InterruptedException,
        RemotingTooMuchRequestException, RemotingTimeoutException, RemotingSendRequestException;

    void invokeOneway(final Channel channel, final RemotingCommand request, final long timeoutMillis)
        throws InterruptedException, RemotingTooMuchRequestException, RemotingTimeoutException,
        RemotingSendRequestException;

}

可以看到两个接口是极为相似的,都有通讯调用方法,注册请求处理器。然后就是作为客户端和服务端各自特有的一些方法。例如producer、consumer获取nameSrv地址列表,服务端注册请求默认处理器,获取本地监听端口等。

客户端对RemotingClient的封装

borker和nameSrv都是直接持有RmotingClient和RemotingServer来进行通讯的,例如BrokerController直接持有RemotingClient对象。客户端producer和consumer则不一样,对RemotingClient进行了两次封装,继承关系如下所示:
RocketMQ是如何通讯的?_第4张图片
从上图可以看到,客户端从RemotingClient封装出了两个对象MQClientAPIImpl和MQClientInstance。
MQClientAPIImpl是对客户端业务的一次封装,将客户端常用的方法提供出来,例如:创建Topic、发送消息、拉取消息查询消息偏移等。RemotingCommand对象到MQClientAPIImpl类中也就截至了,不会再透传到上层。
MQClientInstance类似于简单工厂类,Producer和Consumser对象就是直接持有MQClientInstance的。MQClientInstance提供的功能比较杂,因为它集成了很多模块。到了这一层就更加的业务化了,已经有了broker、namersrv、producer和conumser等概念以及相关的通讯方法了。MQClientInstance我看得也比较累,觉得这一块写的并不是那么的清晰,耦合较重。

底层Netty封装

RocketMQ通讯底层是Netty框架,对其做了简单的封装。基本所有的相关代码都在netty包下面:
RocketMQ是如何通讯的?_第5张图片

NettyRemotingAbstract

RemotingClient和RemotingServer的实现类分别是NettyRemotingClient和NettyRemotingServer,其中公共部分例如网络调用、限流都放在了NettyRmotingAbstract中,通讯的核心类就是NettyRemotingAbstract。相关类的继承关系如下图所示:
RocketMQ是如何通讯的?_第6张图片
下面我们来剖析NettyRemotingAbstract类:

public abstract class NettyRemotingAbstract {
    /**
     * Semaphore to limit maximum number of on-going one-way requests, which protects system memory footprint.
     */
    protected final Semaphore semaphoreOneway;

    /**
     * Semaphore to limit maximum number of on-going asynchronous requests, which protects system memory footprint.
     */
    protected final Semaphore semaphoreAsync;

    /**
     * This map caches all on-going requests.
     */
    protected final ConcurrentMap<Integer /* opaque */, ResponseFuture> responseTable =
        new ConcurrentHashMap<Integer, ResponseFuture>(256);

    /**
     * This container holds all processors per request code, aka, for each incoming request, we may look up the
     * responding processor in this map to handle the request.
     */
    protected final HashMap<Integer/* request code */, Pair<NettyRequestProcessor, ExecutorService>> processorTable =
        new HashMap<Integer, Pair<NettyRequestProcessor, ExecutorService>>(64);

    /**
     * Executor to feed netty events to user defined {@link ChannelEventListener}.
     */
    protected final NettyEventExecutor nettyEventExecutor = new NettyEventExecutor();

    /**
     * The default request processor to use in case there is no exact match in {@link #processorTable} per request code.
     */
    protected Pair<NettyRequestProcessor, ExecutorService> defaultRequestProcessor;
    ......
}

首先是semaphoreOneway和semaphoreAsync两个信号量,用于限流。当高并发调用过于频繁的时候,会抛出RemotingTooMuchRequestException或者RemotingTimeoutException。服务端的相关设置在NettyServerConfig里面,默认值如下:

 private int serverOnewaySemaphoreValue = 256;
 private int serverAsyncSemaphoreValue = 64;

客户端的设置在NettyClientConfig里面,默认值均为65535。一般服务端到客户端的通讯比较少,因此限流值也比客户端少得多。
responseTable 缓存了正在处理的请求,当收到返回的时候会从该表中拿到Future对象处理返回。缓存的key正是创建请求时生成的opaque。有单独的线程会每秒扫描该表,移除超时过期的请求。
processorTable里面存放着注册的处理器,每个requestCode对应一个处理器,一个requestCode就代表了一个业务,例如:
UPDATE_AND_CREATE_TOPIC表示更新或者创建topic。每种业务类型的任务都是递交到线程池去处理的,而每种业务类型都有自己的线程池,这可以带来高并发的优势。
nettyEventExecutor用于处理Netty通信相关事件,不做重点介绍。
defaultRequestProcessor是默认的处理器,在broker端是AdminBrokerProcesssor,在nameSrv端是DefaultRequestProcessor,两者一般都是处理管理功能方面的、与消息发送无关的请求。

请求处理流程

了解上面重要的类之后,下面我们以注册broker信息为例来展示整个通讯流程,加深理解。
Broker通过start()方法启动,并在启动之后主动向NameSrv注册broker信息:

public void start() throws Exception {
        // 省略了部分代码
         if (this.brokerOuterAPI != null) {
            this.brokerOuterAPI.start();
        }
        
        this.registerBrokerAll(true, false);

        // 省略了部分代码
    }
public synchronized void registerBrokerAll(final boolean checkOrderConfig, boolean oneway) {
        TopicConfigSerializeWrapper topicConfigWrapper = this.getTopicConfigManager().buildTopicConfigSerializeWrapper();

        // 省略了部分代码
        RegisterBrokerResult registerBrokerResult = this.brokerOuterAPI.registerBrokerAll(
            this.brokerConfig.getBrokerClusterName(),
            this.getBrokerAddr(),
            this.brokerConfig.getBrokerName(),
            this.brokerConfig.getBrokerId(),
            this.getHAServerAddr(),
            topicConfigWrapper,
            this.filterServerManager.buildNewFilterServerList(),
            oneway,
            this.brokerConfig.getRegisterBrokerTimeoutMills());

        if (registerBrokerResult != null) {
            if (this.updateMasterHAServerAddrPeriodically && registerBrokerResult.getHaServerAddr() != null) {
                this.messageStore.updateHaMasterAddress(registerBrokerResult.getHaServerAddr());
            }

            this.slaveSynchronize.setMasterAddr(registerBrokerResult.getMasterAddr());

            if (checkOrderConfig) {
                this.getTopicConfigManager().updateOrderTopicConfig(registerBrokerResult.getKvTable());
            }
        }
    }

registerAll()方法首先通过topicConfigManager组装需要注册的broker信息,然后通过brokerOuterAPI注册broker信息。其中brokerOuterAPI是封装了remotingClient的一个类,底层就是remotingClient发起请求,如下所示:

public RegisterBrokerResult registerBrokerAll(
        final String clusterName,
        final String brokerAddr,
        final String brokerName,
        final long brokerId,
        final String haServerAddr,
        final TopicConfigSerializeWrapper topicConfigWrapper,
        final List<String> filterServerList,
        final boolean oneway,
        final int timeoutMills) {
        RegisterBrokerResult registerBrokerResult = null;

        List<String> nameServerAddressList = this.remotingClient.getNameServerAddressList();
        if (nameServerAddressList != null) {
            for (String namesrvAddr : nameServerAddressList) {
                try {
                    RegisterBrokerResult result = this.registerBroker(namesrvAddr, clusterName, brokerAddr, brokerName, brokerId,
                        haServerAddr, topicConfigWrapper, filterServerList, oneway, timeoutMills);
                    if (result != null) {
                        registerBrokerResult = result;
                    }

                    log.info("register broker to name server {} OK", namesrvAddr);
                } catch (Exception e) {
                    log.warn("registerBroker Exception, {}", namesrvAddr, e);
                }
            }
        }

        return registerBrokerResult;
    }

首先拿到所有的nameSrv地址,然后循环发起注册broker请求。

 private RegisterBrokerResult registerBroker(
        final String namesrvAddr,
        final String clusterName,
        final String brokerAddr,
        final String brokerName,
        final long brokerId,
        final String haServerAddr,
        final TopicConfigSerializeWrapper topicConfigWrapper,
        final List<String> filterServerList,
        final boolean oneway,
        final int timeoutMills
    ) throws RemotingCommandException, MQBrokerException, RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException,
        InterruptedException {
        RegisterBrokerRequestHeader requestHeader = new RegisterBrokerRequestHeader();
        requestHeader.setBrokerAddr(brokerAddr);
        requestHeader.setBrokerId(brokerId);
        requestHeader.setBrokerName(brokerName);
        requestHeader.setClusterName(clusterName);
        requestHeader.setHaServerAddr(haServerAddr);
        RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.REGISTER_BROKER, requestHeader);

        RegisterBrokerBody requestBody = new RegisterBrokerBody();
        requestBody.setTopicConfigSerializeWrapper(topicConfigWrapper);
        requestBody.setFilterServerList(filterServerList);
        request.setBody(requestBody.encode());

        if (oneway) {
            try {
                this.remotingClient.invokeOneway(namesrvAddr, request, timeoutMills);
            } catch (RemotingTooMuchRequestException e) {
                // Ignore
            }
            return null;
        }

        RemotingCommand response = this.remotingClient.invokeSync(namesrvAddr, request, timeoutMills);
        assert response != null;
        switch (response.getCode()) {
            case ResponseCode.SUCCESS: {
                RegisterBrokerResponseHeader responseHeader =
                    (RegisterBrokerResponseHeader) response.decodeCommandCustomHeader(RegisterBrokerResponseHeader.class);
                RegisterBrokerResult result = new RegisterBrokerResult();
                result.setMasterAddr(responseHeader.getMasterAddr());
                result.setHaServerAddr(responseHeader.getHaServerAddr());
                if (response.getBody() != null) {
                    result.setKvTable(KVTable.decode(response.getBody(), KVTable.class));
                }
                return result;
            }
            default:
                break;
        }

        throw new MQBrokerException(response.getCode(), response.getRemark());
    }

到这里我们就看见了熟悉的RemotingCommand对象了,这里主要就是创建RemotingCommand对象和发起网络调用了。

public RemotingCommand invokeSync(String addr, final RemotingCommand request, long timeoutMillis)
        throws InterruptedException, RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException {
        final Channel channel = this.getAndCreateChannel(addr);
        if (channel != null && channel.isActive()) {
            try {
                if (this.rpcHook != null) {
                    this.rpcHook.doBeforeRequest(addr, request);
                }
                RemotingCommand response = this.invokeSyncImpl(channel, request, timeoutMillis);
                if (this.rpcHook != null) {
                    this.rpcHook.doAfterResponse(RemotingHelper.parseChannelRemoteAddr(channel), request, response);
                }
                return response;
            } catch (RemotingSendRequestException e) {
                log.warn("invokeSync: send request exception, so close the channel[{}]", addr);
                this.closeChannel(addr, channel);
                throw e;
            } catch (RemotingTimeoutException e) {
                if (nettyClientConfig.isClientCloseSocketIfTimeout()) {
                    this.closeChannel(addr, channel);
                    log.warn("invokeSync: close socket because of timeout, {}ms, {}", timeoutMillis, addr);
                }
                log.warn("invokeSync: wait response timeout exception, the channel[{}]", addr);
                throw e;
            }
        } else {
            this.closeChannel(addr, channel);
            throw new RemotingConnectException(addr);
        }
    }

这里主要是创建通讯channer,如果有钩子函数则执行。然后开始Netty调用:

public RemotingCommand invokeSyncImpl(final Channel channel, final RemotingCommand request, final long timeoutMillis)
        throws InterruptedException, RemotingSendRequestException, RemotingTimeoutException {
        final int opaque = request.getOpaque();

        try {
            final ResponseFuture responseFuture = new ResponseFuture(opaque, timeoutMillis, null, null);
            this.responseTable.put(opaque, responseFuture);
            final SocketAddress addr = channel.remoteAddress();
            channel.writeAndFlush(request).addListener(new ChannelFutureListener() {
                @Override
                public void operationComplete(ChannelFuture f) throws Exception {
                    if (f.isSuccess()) {
                        responseFuture.setSendRequestOK(true);
                        return;
                    } else {
                        responseFuture.setSendRequestOK(false);
                    }

                    responseTable.remove(opaque);
                    responseFuture.setCause(f.cause());
                    responseFuture.putResponse(null);
                    PLOG.warn("send a request command to channel <" + addr + "> failed.");
                }
            });

            RemotingCommand responseCommand = responseFuture.waitResponse(timeoutMillis);
            if (null == responseCommand) {
                if (responseFuture.isSendRequestOK()) {
                    throw new RemotingTimeoutException(RemotingHelper.parseSocketAddressAddr(addr), timeoutMillis,
                        responseFuture.getCause());
                } else {
                    throw new RemotingSendRequestException(RemotingHelper.parseSocketAddressAddr(addr), responseFuture.getCause());
                }
            }

            return responseCommand;
        } finally {
            this.responseTable.remove(opaque);
        }
    }

这里会先创建一个ResponseFuture对象,并存放在reponseTable里面,主键就是Reuqest里面的opaque。发起请求后,通过 RemotingCommand responseCommand = responseFuture.waitResponse(timeoutMillis);阻塞线程,等待返回。而返回在哪里处理呢?BrokerOuterAPI启动后,监听返回请求,如下所示:

public void start() {
        this.defaultEventExecutorGroup = new DefaultEventExecutorGroup(//
            nettyClientConfig.getClientWorkerThreads(), //
            new ThreadFactory() {

                private AtomicInteger threadIndex = new AtomicInteger(0);

                @Override
                public Thread newThread(Runnable r) {
                    return new Thread(r, "NettyClientWorkerThread_" + this.threadIndex.incrementAndGet());
                }
            });

        Bootstrap handler = this.bootstrap.group(this.eventLoopGroupWorker).channel(NioSocketChannel.class)//
            .option(ChannelOption.TCP_NODELAY, true)
            .option(ChannelOption.SO_KEEPALIVE, false)
            .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, nettyClientConfig.getConnectTimeoutMillis())
            .option(ChannelOption.SO_SNDBUF, nettyClientConfig.getClientSocketSndBufSize())
            .option(ChannelOption.SO_RCVBUF, nettyClientConfig.getClientSocketRcvBufSize())
            .handler(new ChannelInitializer<SocketChannel>() {
                @Override
                public void initChannel(SocketChannel ch) throws Exception {
                    ch.pipeline().addLast(
                        defaultEventExecutorGroup,
                        new NettyEncoder(),
                        new NettyDecoder(),
                        new IdleStateHandler(0, 0, nettyClientConfig.getClientChannelMaxIdleTimeSeconds()),
                        new NettyConnectManageHandler(),
                        new NettyClientHandler());
                }
            });

        this.timer.scheduleAtFixedRate(new TimerTask() {
            @Override
            public void run() {
                try {
                    NettyRemotingClient.this.scanResponseTable();
                } catch (Exception e) {
                    log.error("scanResponseTable exception", e);
                }
            }
        }, 1000 * 3, 1000);

        if (this.channelEventListener != null) {
            this.nettyEventExecutor.start();
        }
    }
class NettyClientHandler extends SimpleChannelInboundHandler<RemotingCommand> {

        @Override
        protected void channelRead0(ChannelHandlerContext ctx, RemotingCommand msg) throws Exception {
            processMessageReceived(ctx, msg);
        }
    }

然后通过processMessageReceived处理消息,里面会根据请求类型做不同的处理:

 public void processMessageReceived(ChannelHandlerContext ctx, RemotingCommand msg) throws Exception {
        final RemotingCommand cmd = msg;
        if (cmd != null) {
            switch (cmd.getType()) {
                case REQUEST_COMMAND:
                    processRequestCommand(ctx, cmd);
                    break;
                case RESPONSE_COMMAND:
                    processResponseCommand(ctx, cmd);
                    break;
                default:
                    break;
            }
        }
    }

其中REQUEST_COMMAND是处理请求,RESPONSE_COMMAND是处理返回。

 public void processResponseCommand(ChannelHandlerContext ctx, RemotingCommand cmd) {
        final int opaque = cmd.getOpaque();
        final ResponseFuture responseFuture = responseTable.get(opaque);
        if (responseFuture != null) {
            responseFuture.setResponseCommand(cmd);

            responseFuture.release();

            responseTable.remove(opaque);

            if (responseFuture.getInvokeCallback() != null) {
                executeInvokeCallback(responseFuture);
            } else {
                responseFuture.putResponse(cmd);
            }
        } else {
            PLOG.warn("receive response, but not matched any request, " + RemotingHelper.parseChannelRemoteAddr(ctx.channel()));
            PLOG.warn(cmd.toString());
        }
    }

看到了吗?这里通过opaque,就可以轻松的在返回表里找到ResponseFuture,填充返回,并通过putResponse释放锁。这样之前阻塞的线程就可以唤醒继续处理返回。

nameSrv也是同样的处理流程,Netty处理大致相似,这里就从REQUEST_COMMAND开始:

 public void processRequestCommand(final ChannelHandlerContext ctx, final RemotingCommand cmd) {
        final Pair<NettyRequestProcessor, ExecutorService> matched = this.processorTable.get(cmd.getCode());
        final Pair<NettyRequestProcessor, ExecutorService> pair = null == matched ? this.defaultRequestProcessor : matched;
        final int opaque = cmd.getOpaque();

        if (pair != null) {
            Runnable run = new Runnable() {
                @Override
                public void run() {
                    try {
                        RPCHook rpcHook = NettyRemotingAbstract.this.getRPCHook();
                        if (rpcHook != null) {
                            rpcHook.doBeforeRequest(RemotingHelper.parseChannelRemoteAddr(ctx.channel()), cmd);
                        }

                        final RemotingCommand response = pair.getObject1().processRequest(ctx, cmd);
                        if (rpcHook != null) {
                            rpcHook.doAfterResponse(RemotingHelper.parseChannelRemoteAddr(ctx.channel()), cmd, response);
                        }

                        if (!cmd.isOnewayRPC()) {
                            if (response != null) {
                                response.setOpaque(opaque);
                                response.markResponseType();
                                try {
                                    ctx.writeAndFlush(response);
                                } catch (Throwable e) {
                                    PLOG.error("process request over, but response failed", e);
                                    PLOG.error(cmd.toString());
                                    PLOG.error(response.toString());
                                }
                            } else {

                            }
                        }
                    } catch (Throwable e) {
                        PLOG.error("process request exception", e);
                        PLOG.error(cmd.toString());

                        if (!cmd.isOnewayRPC()) {
                            final RemotingCommand response = RemotingCommand.createResponseCommand(RemotingSysResponseCode.SYSTEM_ERROR, //
                                RemotingHelper.exceptionSimpleDesc(e));
                            response.setOpaque(opaque);
                            ctx.writeAndFlush(response);
                        }
                    }
                }
            };

            if (pair.getObject1().rejectRequest()) {
                final RemotingCommand response = RemotingCommand.createResponseCommand(RemotingSysResponseCode.SYSTEM_BUSY,
                    "[REJECTREQUEST]system busy, start flow control for a while");
                response.setOpaque(opaque);
                ctx.writeAndFlush(response);
                return;
            }

            try {
                final RequestTask requestTask = new RequestTask(run, ctx.channel(), cmd);
                pair.getObject2().submit(requestTask);
            } catch (RejectedExecutionException e) {
                if ((System.currentTimeMillis() % 10000) == 0) {
                    PLOG.warn(RemotingHelper.parseChannelRemoteAddr(ctx.channel()) //
                        + ", too many requests and system thread pool busy, RejectedExecutionException " //
                        + pair.getObject2().toString() //
                        + " request code: " + cmd.getCode());
                }

                if (!cmd.isOnewayRPC()) {
                    final RemotingCommand response = RemotingCommand.createResponseCommand(RemotingSysResponseCode.SYSTEM_BUSY,
                        "[OVERLOAD]system busy, start flow control for a while");
                    response.setOpaque(opaque);
                    ctx.writeAndFlush(response);
                }
            }
        } else {
            String error = " request type " + cmd.getCode() + " not supported";
            final RemotingCommand response =
                RemotingCommand.createResponseCommand(RemotingSysResponseCode.REQUEST_CODE_NOT_SUPPORTED, error);
            response.setOpaque(opaque);
            ctx.writeAndFlush(response);
            PLOG.error(RemotingHelper.parseChannelRemoteAddr(ctx.channel()) + error);
        }
    }

首先通过REQUEST_CODE找到处理器,然后将任务包装成RequestTask,并通过线程池去执行任务。具体的执行内容与通讯部分无关,这里就不详细展开了。
总的流程如下所示:
RocketMQ是如何通讯的?_第7张图片

小结:

本来以为选取的RocketMQ通讯这个点已经时比较小的切入点了,没想到内容居然还是这么多。争取下次选取的点再小一点,压缩篇幅,少贴一点代码,多一点思路讲解。
RocketMQ底层通过Netty进行通讯,客户端、Broker、NameSrv根据自己的特点在NettyRemotingAbstract基础上进行了封装。不管是发送请求,还是处理请求,都大量的使用了异步多线程,为高并发提供了强有力的支持。

你可能感兴趣的:(RocketMQ,RocketMQ源码分析系列,RocketMQ,Netty)