远处的海亲吻着天

RocketMQ 源码分析

NameServer

架构设计

启动流程

Step1

Step2

Step3

NameServer路由管理

路由元信息

路由注册

路由删除

路由发现

NameServer小结

Producer

方法和属性

启动流程

消息发送

选择消息队列

消息发送

NameServer

架构设计

Broker消息服务器在启动时向所有NameServer注册,消息生产者(Producer)在发送消息之前先从NameServer获取Broker服务器地址列表,然后根据负载均衡算法从列表中选择一台服务器进行消息发送。NameServer与每台Broker保持长连接,并间隔10s检测Broker是否存活,如果检测到Broker宕机,则从路由注册表中将其移除。但是路由变化不会马上通知消息生产者,这是为了降低NameServer实现的复杂性,在消息发送端提供容错机制来保证消息发送的高可用性。

NameServer本身的高可用可通过部署多台NameServer服务器来实现,但彼此之间互不通信,也就是NameServer服务器之间在某一时刻的数据并不会完全相同,但这对消息发送不会造成任何影响,这也是RocketMQ NameServer设计的一个亮点,RocketMQ NameServer设计追求简单高效。

启动流程

NameServer启动类: org.apache.rocketmq.namesrv.NamesrvStartup

Step1

首先来解析配置文件,需要填充NamesrvConfig、NettyServerConfig属性值。

            final NamesrvConfig namesrvConfig = new NamesrvConfig();
            final NettyServerConfig nettyServerConfig = new NettyServerConfig();
            nettyServerConfig.setListenPort(9876);
            if (commandLine.hasOption('c')) {
                String file = commandLine.getOptionValue('c');
                if (file != null) {
                    InputStream in = new BufferedInputStream(new FileInputStream(file));
                    properties = new Properties();
                    properties.load(in);
                    MixAll.properties2Object(properties, namesrvConfig);
                    MixAll.properties2Object(properties, nettyServerConfig);

                    namesrvConfig.setConfigStorePath(file);

                    System.out.printf("load config properties file OK, " + file + "%n");
                    in.close();
                }
            }

            if (commandLine.hasOption('p')) {
                MixAll.printObjectProperties(null, namesrvConfig);
                MixAll.printObjectProperties(null, nettyServerConfig);
                System.exit(0);
            }

            MixAll.properties2Object(ServerUtil.commandLine2Properties(commandLine), namesrvConfig);

从代码我们可以知道先创建NamesrvConfig(NameServer业务参数)、NettyServerConfig(NameServer网络参数),然后在解析启动时把指定的配置文件或启动命令中的选项值,填充到namesrvConfig,nettyServerConfig对象。
参数来源有如下两种方式:
1、-c configFile通过-c命令指定配置文件的路径
2、使用"--属性名属性值",例如--listenPort 9876

NameServerConfig属性

    private String rocketmqHome = System.getProperty(MixAll.ROCKETMQ_HOME_PROPERTY, System.getenv(MixAll.ROCKETMQ_HOME_ENV));
    private String kvConfigPath = System.getProperty("user.home") + File.separator + "namesrv" + File.separator + "kvConfig.json";
    private String configStorePath = System.getProperty("user.home") + File.separator + "namesrv" + File.separator + "namesrv.properties";
    private String productEnvName = "center";
    private boolean clusterTest = false;
    private boolean orderMessageEnable = false;

rocketmqhome: rocketmq主目录,可以通过-Drocketmq.home.dir=path或通过设置环境变量ROCKETMQ_HOME来配置RocketMQ的主目录。
kvConfigPath: NameServer存储KV配置属性的持久化路径。
configStorePath: nameServer默认配置文件路径,不生效。nameServer启动时如果要通过配置文件配置NameServer启动属性的话,请使用-c选项。
orderMessageEnable: 是否支持顺序消息,默认是不支持

NettyServerConfig属性

    private int listenPort = 8888;
    private int serverWorkerThreads = 8;
    private int serverCallbackExecutorThreads = 0;
    private int serverSelectorThreads = 3;
    private int serverOnewaySemaphoreValue = 256;
    private int serverAsyncSemaphoreValue = 64;
    private int serverChannelMaxIdleTimeSeconds = 120;
    private int serverSocketSndBufSize = NettySystemConfig.socketSndbufSize;
    private int serverSocketRcvBufSize = NettySystemConfig.socketRcvbufSize;
    private boolean serverPooledByteBufAllocatorEnable = true;
    private boolean useEpollNativeSelector = false;

listenPort: NameServer监听端口,该值默认会被初始化9876。
serverWorkerThreads: Netty业务线程池线程个数。
serverCallbackExecutorThreads: Netty public 任务线程池线程个数,Netty网络设计,根据业务类型会常见不同的线程池,比如处理消息发送、消息消费、心跳检测等等。如果该业务类型(RequestCode)未注册线程池,则由public线程池执行。
serverSelectorThreads:IO线程池线程个数,主要是NameServer、Broker端解析请求、返回相应的线程个数,这类线程主要是处理网络请求的,解析请求包,然后转发到各个业务线程池完成具体的业务操作,然后将结果返回调用方。
serverOnewaySemaphoreValue: send oneway消息请求并发度(Broker端参数)。
serverAsyncSemaphoreValue: 异步消息发送最大并发度(Broker端参数)。
serverChannelMaxIdleTimeSeconds: 网络连接最大空闲时间,默认120s。如果连接空闲时间超过该参数设置的值,连接将被关闭。
serverSocketSndBufSize: 网络socket发送缓存区大小,默认64k。
serverSocketRecBufSize: 网络socket接收缓存区大小,默认64k。
serverPooledByteBufAllocatorEnable: ByteBuffer是否开启缓存,建议开启。
useEpollNativeSelector: 是否启用Epoll IO模型,Linux环境建议开启。

在启动NameServer时,可以先使用./mqnameserver -c configFile -p 打印当前加载的配置属性

Step2

根据启动属性创建NamesrvController实例,并初始化该实例,NamesrvController实例为NameServer核心控制器。

NamesrvController#initialize代码片段

    public boolean initialize() {

        this.kvConfigManager.load();

        this.remotingServer = new NettyRemotingServer(this.nettyServerConfig, this.brokerHousekeepingService);

        this.remotingExecutor =
            Executors.newFixedThreadPool(nettyServerConfig.getServerWorkerThreads(), new ThreadFactoryImpl("RemotingExecutorThread_"));

        this.registerProcessor();

        this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {

            @Override
            public void run() {
                NamesrvController.this.routeInfoManager.scanNotActiveBroker();
            }
        }, 5, 10, TimeUnit.SECONDS);

        this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {

            @Override
            public void run() {
                NamesrvController.this.kvConfigManager.printAllPeriodically();
            }
        }, 1, 10, TimeUnit.MINUTES);

        return true;
    }

加载KV配置,创建NettyServer网络处理对象,然后开启两个定时任务,在RocketMQ中此类定时任务统称为心跳检测。

定时任务1: NameServer每隔10s扫描一次Broker,移除处于不激活状态的Broker。
定时任务2: NameServer每隔十分钟打印一次KV配置。

Step3

注册JVM钩子函数并启动服务器,以便监听Broker、消息生产者的网络请求

            Runtime.getRuntime().addShutdownHook(new ShutdownHookThread(log, new Callable() {
                @Override
                public Void call() throws Exception {
                    controller.shutdown();
                    return null;
                }
            }));

            controller.start();

NameServer路由管理

NameServer主要作用是为消息生产者和消息消费者提供关于主题Topic的路由信息,那么NameServer需要存储路由的基础信息,还要能够管理Broker节点,包括路由注册、路由删除等功能。

路由元信息

NameServer路由实现类: org.apache.rocketmq.namesrv.routeinfo.RouteInfoManager,在了解路由注册之前,我们首先看一下NameServer到底存储哪些信息。

    private final HashMap> topicQueueTable;
    private final HashMap brokerAddrTable;
    private final HashMap> clusterAddrTable;
    private final HashMap brokerLiveTable;
    private final HashMap/* Filter Server */> filterServerTable;

topicQueueTable: Topic消息队列路由信息,消息发送时根据路由表进行负载均衡。
brokerAddrTable: Broker基础信息,包含BrokerName、所属集群名称、主备Broker地址。
clusterAddrTable: Broker集群信息,存储集群中所有Broker名称。
brokerLiveTable: Broker状态信息。NameServer每次收到心跳包时会替换该信息。
filterServerTable: Broker上的FilterServer列表,用于类模式消息过滤

RocketMQ基于订阅发布机制,一个Topic拥有多个消息队列,一个Broker为每一主题默认创建4个读队列4个写队列。多个Broker组成一个集群,集群由相同的多台Broker组成Master-Slave架构,brokerId为0表示Master,大于0表示Slave。BrokerLiveInfo中的lastUpdateTimestamp存储上次收到Broker心跳包的时间。

路由注册

RocketMQ路由注册是通过Broker与NameServer的心跳功能实现的。Broker启动时向集群中所有的NameServer发送心跳语句,每隔30秒向集群中所有NameServer发送心跳包,NameServer收到Broker心跳包时会更新brokerLiveTable缓存中BrokerLiveInfo的lastUpdateTimestamp,然后NameServer每隔10秒扫描brokerLiveTable,如果连续120s没有收到心跳包,NameServer将移除该Broker的路由信息同时关闭Socket连接。

Broker端心跳包发送(BrokerController#start)

        this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
            @Override
            public void run() {
                try {
                    BrokerController.this.registerBrokerAll(true, false);
                } catch (Throwable e) {
                    log.error("registerBrokerAll Exception", e);
                }
            }
        }, 1000 * 10, 1000 * 30, TimeUnit.MILLISECONDS);

BrokerOuterAPI#registerBrokerAll

        List nameServerAddressList = this.remotingClient.getNameServerAddressList();
        if (nameServerAddressList != null) {
            for (String namesrvAddr : nameServerAddressList) {// 遍历所有NameServer列表
                try {
                    RegisterBrokerResult result = this.registerBroker(namesrvAddr, clusterName, brokerAddr, brokerName, brokerId,
                        haServerAddr, topicConfigWrapper, filterServerList, oneway, timeoutMills);// 分别向NameServer注册
                    if (result != null) {
                        registerBrokerResult = result;
                    }

                    log.info("register broker to name server {} OK", namesrvAddr);
                } catch (Exception e) {
                    log.warn("registerBroker Exception, {}", namesrvAddr, e);
                }
            }
        }

BrokerOuteAPI#registerBroker(网络发送代码)

        RegisterBrokerRequestHeader requestHeader = new RegisterBrokerRequestHeader();
        requestHeader.setBrokerAddr(brokerAddr);
        requestHeader.setBrokerId(brokerId);
        requestHeader.setBrokerName(brokerName);
        requestHeader.setClusterName(clusterName);
        requestHeader.setHaServerAddr(haServerAddr);
        RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.REGISTER_BROKER, requestHeader);

        RegisterBrokerBody requestBody = new RegisterBrokerBody();
        requestBody.setTopicConfigSerializeWrapper(topicConfigWrapper);
        requestBody.setFilterServerList(filterServerList);
        request.setBody(requestBody.encode());

        if (oneway) {
            try {
                this.remotingClient.invokeOneway(namesrvAddr, request, timeoutMills);
            } catch (RemotingTooMuchRequestException e) {
                // Ignore
            }
            return null;
        }

        RemotingCommand response = this.remotingClient.invokeSync(namesrvAddr, request, timeoutMills);

发送心跳包具体逻辑,首先封装请求包头(Header)。

brokerAddr: broker地址。
brokerId: brokerId,0: Master ; 大于0: Slave。
brokerName: broker名称。
clusterName: 集群名称。
haServerAddr: master地址,初次请求时该值为空,slave向Nameserver注册后返回。
requestBody:
- filterServerList: 消息过滤服务器列表。
- topicConfigWapper: 主题配置,topicConfigWrapper内部封装的是TopicConfigManager中的topicConfigTable,内部存储的是Broker启动时默认的一些Topic,MixAll.SELF_TEST_TOPIC、MixAll.DEFAULT_TOPICC(AutoCreateTopicEnable=true)、MixAll.BENCHMARK_TOPIC、MixAll.OFFSET_MOVED_EVENT、BrokerConfig#brokerClusterName、BrokerConfig#brokerName。Broker中Topic默认存储在${Rocket_Home}/store/config/topics.json中。

NameServer处理心跳包

org.apache.rocketmq.namesrv.processor.DefaultRequestProcessor网络处理器解析请求类型,如果请求类型为RequestCode.REGISTER_BROKER,则请求最终转发到RouteInfoManager#registerBroker。

Step1: 路由注册需要加写锁,防止并发修改RouteInfoManager中的路由表。首先判断Broker所属集群是否存在,如果不存在,则创建,然后将broker名加入到集群Broker集合中。

RouteInfoManager#registerBroker clusterAddrTable维护

                this.lock.writeLock().lockInterruptibly();
                Set brokerNames = this.clusterAddrTable.get(clusterName);
                if (null == brokerNames) {
                    brokerNames = new HashSet();
                    this.clusterAddrTable.put(clusterName, brokerNames);
                }
                brokerNames.add(brokerName);

Step2: 维护BrokerData信息,首先从brokerAddrTable根据BrokerName尝试获取Broker信息,如果不存在,则新建BrokerData并放入brokerAddrTable,registerFirst设置为true;如果存在,直接替换原先的,registerFirst设置为false,表示非第一次注册

RouteInfoManager#registerBroker brokerAddrTable维护

                BrokerData brokerData = this.brokerAddrTable.get(brokerName);
                if (null == brokerData) {
                    registerFirst = true;
                    brokerData = new BrokerData(clusterName, brokerName, new HashMap());
                    this.brokerAddrTable.put(brokerName, brokerData);
                }
                String oldAddr = brokerData.getBrokerAddrs().put(brokerId, brokerAddr);
                registerFirst = registerFirst || (null == oldAddr);

Step3: 如果Broker为Master,并且Broker Topic配置信息发变化或者是初次注册,则需要创建或更新Topic路由元数据,填充topicQueueTable,其实就是为默认主题自动注册路由信息,其中包含MixAll.DEFAULT_TOPIC的路由信息。当消费生产者发送主题时,如果该主题未创建并且BrokerConfig的autoCreateTopicEnable为true时,则返回MixAll.DEFAULT_TOPIC的路由信息

                if (null != topicConfigWrapper
                    && MixAll.MASTER_ID == brokerId) {
                    if (this.isBrokerTopicConfigChanged(brokerAddr, topicConfigWrapper.getDataVersion())
                        || registerFirst) {
                        ConcurrentMap tcTable =
                            topicConfigWrapper.getTopicConfigTable();
                        if (tcTable != null) {
                            for (Map.Entry entry : tcTable.entrySet()) {
                                this.createAndUpdateQueueData(brokerName, entry.getValue());
                            }
                        }
                    }
                }

根据TopicConfig创建QueueData数据结构,然后更新topicQueueTable
RouteInfoManager#createAndUpdateQueueData

    private void createAndUpdateQueueData(final String brokerName, final TopicConfig topicConfig) {
        QueueData queueData = new QueueData();
        queueData.setBrokerName(brokerName);
        queueData.setWriteQueueNums(topicConfig.getWriteQueueNums());
        queueData.setReadQueueNums(topicConfig.getReadQueueNums());
        queueData.setPerm(topicConfig.getPerm());
        queueData.setTopicSynFlag(topicConfig.getTopicSysFlag());

        List queueDataList = this.topicQueueTable.get(topicConfig.getTopicName());
        if (null == queueDataList) {
            queueDataList = new LinkedList();
            queueDataList.add(queueData);
            this.topicQueueTable.put(topicConfig.getTopicName(), queueDataList);
            log.info("new topic registered, {} {}", topicConfig.getTopicName(), queueData);
        } else {
            boolean addNewOne = true;

            Iterator it = queueDataList.iterator();
            while (it.hasNext()) {
                QueueData qd = it.next();
                if (qd.getBrokerName().equals(brokerName)) {
                    if (qd.equals(queueData)) {
                        addNewOne = false;
                    } else {
                        log.info("topic changed, {} OLD: {} NEW: {}", topicConfig.getTopicName(), qd,
                            queueData);
                        it.remove();
                    }
                }
            }

            if (addNewOne) {
                queueDataList.add(queueData);
            }
        }
    }

Step4: 更新BrokerLiveInfo,存活Broker信息表,BrokerLiveInfo是执行路由删除的重要依据

RouteInfoManager#registerBroker

                BrokerLiveInfo prevBrokerLiveInfo = this.brokerLiveTable.put(brokerAddr,
                    new BrokerLiveInfo(
                        System.currentTimeMillis(),
                        topicConfigWrapper.getDataVersion(),
                        channel,
                        haServerAddr));
                if (null == prevBrokerLiveInfo) {
                    log.info("new broker registered, {} HAServer: {}", brokerAddr, haServerAddr);
                }

Step5: 注册Broker的过滤器Server地址列表,一个Broker上会关联多个FilterServer消息过滤服务器;如果此Broker为从节点,则需要查找该Broker的Master的节点信息,并更新对应的masterAddr属性。

                if (filterServerList != null) {
                    if (filterServerList.isEmpty()) {
                        this.filterServerTable.remove(brokerAddr);
                    } else {
                        this.filterServerTable.put(brokerAddr, filterServerList);
                    }
                }

                if (MixAll.MASTER_ID != brokerId) {
                    String masterAddr = brokerData.getBrokerAddrs().get(MixAll.MASTER_ID);
                    if (masterAddr != null) {
                        BrokerLiveInfo brokerLiveInfo = this.brokerLiveTable.get(masterAddr);
                        if (brokerLiveInfo != null) {
                            result.setHaServerAddr(brokerLiveInfo.getHaServerAddr());
                            result.setMasterAddr(masterAddr);
                        }
                    }
                }

路由删除

broker每隔30秒向集群中所有NameServer发送心跳包,心跳包包含brokerId,broker地址，broker名称,broker集群名称、broker关联的filtersever列表。但是如果broker宕机，namesever无法收到心跳包，此时namesever如何剔除这些失效的broker呢？NameServer会每隔10s扫描brokerLiveTable状态表,如果BrokerLive的lastUpdateTimestamp的时间戳距当前时间超过120s,则认为Broker失效,移除该Broker,关闭与Broker连接,并同时更新topicQueueTable、brokerAddrTable、brokerLiveTable、filterServerTable。

RocketMQ有两个触发点来出发路由删除。
1、NameServer定时扫描brokerLiveTable检测上次心跳包与当前系统时间的时间差,如果时间戳大于120s,则需要移除该Broker信息。
2、Broker在正常被关闭的情况下,会执行unregisterBroker指令。
由于不管是何种方式出发的路由删除,路由删除的方法都是一样的,就是从topicQueueTable、brokerAddrTable、brokerLiveTable、filterServerTable删除与该Broker相关的信息。

RouteInfoManager#scanNotActiveBroker

    public void scanNotActiveBroker() {
        Iterator> it = this.brokerLiveTable.entrySet().iterator();
        while (it.hasNext()) {
            Entry next = it.next();
            long last = next.getValue().getLastUpdateTimestamp();
            if ((last + BROKER_CHANNEL_EXPIRED_TIME) < System.currentTimeMillis()) {
                RemotingUtil.closeChannel(next.getValue().getChannel());
                it.remove();
                log.warn("The broker channel expired, {} {}ms", next.getKey(), BROKER_CHANNEL_EXPIRED_TIME);
                this.onChannelDestroy(next.getKey(), next.getValue().getChannel());
            }
        }
    }

遍历brokerLiveInfo路由表(HashMap),检测BrokerLiveInfo的lastUpdateTimestamp上次收到心跳包的时间如果超过当前时间120s,NameServer则认为该Broker已不可用,故需要将它移除,关闭Channel,然后删除与该Broker相关的路由信息,路由表维护过程,需要申请写锁。

Step1: 申请写锁,根据brokerAddress从brokerLiveTable、filterServerTable移除

RouteInfoManager#onChannelDestroy

                    this.lock.writeLock().lockInterruptibly();
                    this.brokerLiveTable.remove(brokerAddrFound);
                    this.filterServerTable.remove(brokerAddrFound);

Step2: 维护brokerAddrTable。遍历从HashMap brokerAddrTable,从BrokerData的HashMap brokerAddrs中,找到具体的Broker,从BrokerData中移除,如果移除后在BrokerData中不再包含其他Broker,则在brokerAddrTable中移除该brokerName对应的条目。

RouteInfoManager#onChannelDestroy

                    String brokerNameFound = null;
                    boolean removeBrokerName = false;
                    Iterator> itBrokerAddrTable =
                        this.brokerAddrTable.entrySet().iterator();
                    while (itBrokerAddrTable.hasNext() && (null == brokerNameFound)) {
                        BrokerData brokerData = itBrokerAddrTable.next().getValue();

                        Iterator> it = brokerData.getBrokerAddrs().entrySet().iterator();
                        while (it.hasNext()) {
                            Entry entry = it.next();
                            Long brokerId = entry.getKey();
                            String brokerAddr = entry.getValue();
                            if (brokerAddr.equals(brokerAddrFound)) {
                                brokerNameFound = brokerData.getBrokerName();
                                it.remove();
                                log.info("remove brokerAddr[{}, {}] from brokerAddrTable, because channel destroyed",
                                    brokerId, brokerAddr);
                                break;
                            }
                        }

                        if (brokerData.getBrokerAddrs().isEmpty()) {
                            removeBrokerName = true;
                            itBrokerAddrTable.remove();
                            log.info("remove brokerName[{}] from brokerAddrTable, because channel destroyed",
                                brokerData.getBrokerName());
                        }

Step3: 根据BrokerName,从clusterAddrTable中找到Broker并从集群中移除。如果移除后,集群中不包含任何Broker,则将该集群从clusterAddrTable中移除。

RouteInfoManager#onChannelDestroy

                    if (brokerNameFound != null && removeBrokerName) {
                        Iterator>> it = this.clusterAddrTable.entrySet().iterator();
                        while (it.hasNext()) {
                            Entry> entry = it.next();
                            String clusterName = entry.getKey();
                            Set brokerNames = entry.getValue();
                            boolean removed = brokerNames.remove(brokerNameFound);
                            if (removed) {
                                log.info("remove brokerName[{}], clusterName[{}] from clusterAddrTable, because channel destroyed",
                                        brokerNameFound, clusterName);

                                if (brokerNames.isEmpty()) {
                                    log.info("remove the clusterName[{}] from clusterAddrTable, because channel destroyed and no broker in this cluster",
                                            clusterName);
                                    it.remove();
                                }

                                break;
                            }
                        }
                    }

Step4: 根据brokerName,遍历所有主题的队列,如果队列中包含了当前Broker的队列,则移除,如果topic只包含待移除Broker的队列的话,从路由表中删除该topic。

RouteInfoManager#onChannelDestroy

                    if (removeBrokerName) {
                        Iterator>> itTopicQueueTable =
                                this.topicQueueTable.entrySet().iterator();
                        while (itTopicQueueTable.hasNext()) {
                            Entry> entry = itTopicQueueTable.next();
                            String topic = entry.getKey();
                            List queueDataList = entry.getValue();

                            Iterator itQueueData = queueDataList.iterator();
                            while (itQueueData.hasNext()) {
                                QueueData queueData = itQueueData.next();
                                if (queueData.getBrokerName().equals(brokerNameFound)) {
                                    itQueueData.remove();
                                    log.info("remove topic[{} {}], from topicQueueTable, because channel destroyed",
                                            topic, queueData);
                                }
                            }

                            if (queueDataList.isEmpty()) {
                                itTopicQueueTable.remove();
                                log.info("remove topic[{}] all queue, from topicQueueTable, because channel destroyed",
                                        topic);
                            }
                        }
                    }

Step5: 释放锁,完成路由删除。

RouteInfoManager#onChannelDestroy

                finally {
                    this.lock.writeLock().unlock();
                }

路由发现

RocketMQ路由发现是非实时的,当Topic路由出现变化后,NameServer不主动推送给客户端,而是由客户端定时拉取主题最新的路由。根据主题名称拉取路由信息的命令编码为: GET_ROUTEINFO_BY_TOPIC。

private String orderTopicConf;
private List queueDatas;
private List brokerDatas;
private HashMap/* Filter Server */> filterServerTable;

orderTopicConf: 顺序消息配置内容,来自于kvConfig。
List queueDatas: topic队列元数据
List brokerDatas: topic分布的broker元数据
HashMap/, List/ Filter Server */> filterServerTable: broker上过滤服务器地址列表。

NameServer路由发现实现类: DefaultRequestProcessor#getRouteInfoByTopic

DefaultRequestProcessor#getRouteInfoByTopic

    public RemotingCommand getRouteInfoByTopic(ChannelHandlerContext ctx,
        RemotingCommand request) throws RemotingCommandException {
        final RemotingCommand response = RemotingCommand.createResponseCommand(null);
        final GetRouteInfoRequestHeader requestHeader =
            (GetRouteInfoRequestHeader) request.decodeCommandCustomHeader(GetRouteInfoRequestHeader.class);

        TopicRouteData topicRouteData = this.namesrvController.getRouteInfoManager().pickupTopicRouteData(requestHeader.getTopic());

        if (topicRouteData != null) {
            if (this.namesrvController.getNamesrvConfig().isOrderMessageEnable()) {
                String orderTopicConf =
                    this.namesrvController.getKvConfigManager().getKVConfig(NamesrvUtil.NAMESPACE_ORDER_TOPIC_CONFIG,
                        requestHeader.getTopic());
                topicRouteData.setOrderTopicConf(orderTopicConf);
            }

            byte[] content = topicRouteData.encode();
            response.setBody(content);
            response.setCode(ResponseCode.SUCCESS);
            response.setRemark(null);
            return response;
        }

        response.setCode(ResponseCode.TOPIC_NOT_EXIST);
        response.setRemark("No topic route info in name server for the topic: " + requestHeader.getTopic()
            + FAQUrl.suggestTodo(FAQUrl.APPLY_TOPIC_URL));
        return response;
    }

Step1: 调用RouterInfoManager的方法,从路由表topicQueueTable、brokerAddrTable、filterServerTable中分别填充TopicRouteData中的List、List和filterServer地址表。
Step2: 如果找到主题对应的路由信息并且该主题为顺序消息,则从NameServer KVconfig中获取关于顺序消息相关的配置填充路由信息。如果找不到路由信息CODE则使用TOPIC_NOT_EXISTS,表示没有找到对应的路由。

NameServer小结

Producer

消息生产者的代码都在client模块中，相对RocketMQ来讲，消息生产者就是客户端，也是消息提供者

方法和属性

主要方法介绍

    //创建主题 
    void createTopic(final String key, final String newTopic, final int queueNum)
        throws MQClientException;
    
    //根据时间戳从队列中查找消息偏移量
    long searchOffset(final MessageQueue mq, final long timestamp) throws MQClientException;

    //查找消息队列中最大偏移量
    long maxOffset(final MessageQueue mq) throws MQClientException;

    //查找消息队列中最小偏移量
    long minOffset(final MessageQueue mq) throws MQClientException;

    //根据偏移量寻找消息
    MessageExt viewMessage(final String offsetMsgId) throws RemotingException, MQBrokerException,
        InterruptedException, MQClientException;

    //根据条件查找消息
    QueryResult queryMessage(final String topic, final String key, final int maxNum, final long begin,
        final long end) throws MQClientException, InterruptedException;

    //根据消息id和主题查找消息
    MessageExt viewMessage(String topic,
        String msgId) throws RemotingException, MQBrokerException, InterruptedException, MQClientException;

    //启动
    void start() throws MQClientException;
        
    //关闭
    void shutdown();

    //查找该主题下的所有消息
    List fetchPublishMessageQueues(final String topic) throws MQClientException;
    
    //同步发送消息
    SendResult send(final Message msg) throws MQClientException, RemotingException, MQBrokerException,
        InterruptedException;

    //同步超时发送消息
    SendResult send(final Message msg, final long timeout) throws MQClientException,
        RemotingException, MQBrokerException, InterruptedException;

    //异步发送消息
    void send(final Message msg, final SendCallback sendCallback) throws MQClientException,
        RemotingException, InterruptedException;

    //异步超时发送消息
    void send(final Message msg, final SendCallback sendCallback, final long timeout)
        throws MQClientException, RemotingException, InterruptedException;

    //发送单向消息
    void sendOneway(final Message msg) throws MQClientException, RemotingException,
        InterruptedException;

    //选择指定队列同步发送消息
    SendResult send(final Message msg, final MessageQueue mq) throws MQClientException,
        RemotingException, MQBrokerException, InterruptedException;
    
    //选择指定队列单项发送消息
    void sendOneway(final Message msg, final MessageQueue mq) throws MQClientException,
        RemotingException, InterruptedException;

    //批量发送消息
    SendResult send(final Collection msgs) throws MQClientException, RemotingException, MQBrokerException,
        InterruptedException;

属性介绍

producerGroup: 生产者所属组,消息服务器在回查事务状态时会随机选择该组中任何一个生产者发起事务回查请求。

createTopicKey: 默认topicKey。

defaultTopicQueueNums: 默认主题在每一个Broker队列数量。

sendMsgTimeout: 发送消息默认超时时间,默认3s。

compressMsgBodyOverHowmuch: 消息体超过该值则启用压缩,默认4K。

retryTimesWhenSendFailed: 同步方式发送消息重试次数,默认为2,总共执行3次。

retryTimesWhenSendAsyncFailed: 异步方式发送消息重试次数,默认为2。

retryAnotherBrokerWhenNotStoreOK: 消息重试时选择另外一个Broker时,是否不等储存结果就返回,默认为false。

maxMessageSize: 允许发送的最大消息长度,默认为4M,该值最大值为2^32-1。

启动流程

我们可以从DefaultMQProducerImpl的start方法来追踪,具体细节如下。

Step1:检查producerGroup是否符合要求;并改变生产者的instanceName为进程ID。

DefaultMQProducerImpl#start

//检查producergroup名称格式等
this.checkConfig();
//设置生产这instanceName为进程ID
if (!this.defaultMQProducer.getProducerGroup().equals(MixAll.CLIENT_INNER_PRODUCER_GROUP)) {
    this.defaultMQProducer.changeInstanceNameToPID();
}

Step2:创建MQClientInstance实例。整个JVM实例中只存在一个MQClientManager实例,维护一个MQClientInstance缓存表private ConcurrentMap factoryTable =new ConcurrentHashMap(),也就是同一个clientId只会创建一个MQClientInstance。MQClientInstance封装了RocketMQ网络处理api，是消息生产者和消息消费者与namesever、broker打交道的网络通道

DefaultMQProducerImpl#start

this.mQClientFactory = MQClientManager.getInstance().getAndCreateMQClientInstance(this.defaultMQProducer, rpcHook);

MQClientManager#getAndCreateMQClientInstance

public MQClientInstance getAndCreateMQClientInstance(final ClientConfig clientConfig, RPCHook rpcHook) {
      String clientId = clientConfig.buildMQClientId();
      MQClientInstance instance = this.factoryTable.get(clientId);
      if (null == instance) {
          instance =
              new MQClientInstance(clientConfig.cloneClientConfig(),
                  this.factoryIndexGenerator.getAndIncrement(), clientId, rpcHook);
          MQClientInstance prev = this.factoryTable.putIfAbsent(clientId, instance);
          if (prev != null) {
              instance = prev;
              log.warn("Returned Previous MQClientInstance for clientId:[{}]", clientId);
          } else {
              log.info("Created new MQClientInstance for clientId:[{}]", clientId);
          }
      }

      return instance;
}

org.apache.rocketmq.client.ClientConfig#buildMQClientId

// 创建clientId的方法
public String buildMQClientId() {
    StringBuilder sb = new StringBuilder();
    sb.append(this.getClientIP());

    sb.append("@");
    sb.append(this.getInstanceName());
    if (!UtilAll.isBlank(this.unitName)) {
        sb.append("@");
        sb.append(this.unitName);
    }

    return sb.toString();
}

clientId为客户端IP+instance+(unitName可选),如果在同一台服务器部署两个应用程序,应用程序岂不是clientId相同,会造成混乱?

为了避免这个问题,如果instance为默认值DEFAULT的话,RocketMQ会自动将instance设置为进程ID,这样避免了不同进程的相互影响,但同一个JVM中的不同消费者和不同生产者在启动时获取到的MQClientInstance实例都是同一个。根据后面的介绍,MQClientInstance封装了RocketMQ网络处理API,是消息生产者(Producer)、消息消费者(Consumer)与NameServer、Broker打交道的网络通道。

Step3:向MQClientInstance注册,将当前生产者加入到MQClientInstance管理中,方便后续调用网络请求、进行心跳检测等。

boolean registerOK = mQClientFactory.registerProducer(this.defaultMQProducer.getProducerGroup(), this);
if (!registerOK) {
    this.serviceState = ServiceState.CREATE_JUST;
    throw new MQClientException("The producer group[" + this.defaultMQProducer.getProducerGroup()
       + "] has been created before, specify another name please." + FAQUrl.suggestTodo(FAQUrl.GROUP_NAME_DUPLICATE_URL),null);
}

Step4:启动MQClientInstance,如果MQClientInstance已经启动,则本次启动不会真正执行。

消息发送

消息发送流程主要的步骤: 验证消息、查找路由、消息发送(包含异常处理机制)。

同步消息发送入口,代码如下

// DefaultMQProducer#send
public SendResult send(
    Message msg) throws MQClientException, RemotingException, MQBrokerException, InterruptedException {
    return this.defaultMQProducerImpl.send(msg);
}
// DefaultMQProducerImpl#send
public SendResult send(
    Message msg) throws MQClientException, RemotingException, MQBrokerException, InterruptedException {
    return send(msg, this.defaultMQProducer.getSendMsgTimeout());
}

默认消息发送以同步方式发送,默认超时时间为3s。

消息长度验证

DefaultMQProducerImpl#sendDefaultImp

消息发送之前,首先确保生产者处于运行状态,然后验证消息是否符合相应的规范,具体的规范要求是主题名称、消息体不能为空、消息长度不能等于0且默认不能超过允许发送消息的最大长度4M(maxMessageSize = 1024 * 1024 * 4)。

查找主题路由信息

消息发送之前,首先需要获取主题的路由信息,只有获取了这些信息我们才能知道消息发送到具体的Broker节点。
DefaultMQProducerImpl#tryToFindTopicPublishInfo

private TopicPublishInfo tryToFindTopicPublishInfo(final String topic) {
    TopicPublishInfo topicPublishInfo = this.topicPublishInfoTable.get(topic);
    if (null == topicPublishInfo || !topicPublishInfo.ok()) {
        this.topicPublishInfoTable.putIfAbsent(topic, new TopicPublishInfo());
        this.mQClientFactory.updateTopicRouteInfoFromNameServer(topic);
        topicPublishInfo = this.topicPublishInfoTable.get(topic);
    }

    if (topicPublishInfo.isHaveTopicRouterInfo() || topicPublishInfo.ok()) {
        return topicPublishInfo;
    } else {
        this.mQClientFactory.updateTopicRouteInfoFromNameServer(topic, true, this.defaultMQProducer);
        topicPublishInfo = this.topicPublishInfoTable.get(topic);
        return topicPublishInfo;
    }
}

tryToFindTopicPublishInfo是查找主题的路由信息的方法。如果生产者中缓存了topic的路由信息,如果该路由信息中包含了消息队列,则直接返回该路由信息,如果没有缓存或没有消息队列,则向NameServer查询该Topic的路由信息。如果最终未找到路由信息,则抛出异常:无法找到主题相关路由信息异常

先看一下TopicPublishInfo

public class TopicPublishInfo {
    private boolean orderTopic = false;
    private boolean haveTopicRouterInfo = false;
    private List messageQueueList = new ArrayList();
    private volatile ThreadLocalIndex sendWhichQueue = new ThreadLocalIndex();
    private TopicRouteData topicRouteData;
}

public class TopicRouteData extends RemotingSerializable {
    private String orderTopicConf;
    private List queueDatas;
    private List brokerDatas;
    private HashMap/* Filter Server */> filterServerTable;
}

TopicPublishInfo的属性:

orderTopic: 是否是顺序消息
List messageQueueList: 该主题队列的消息队列
sendWhichQueue : 每选择一次消息队列,该值会自增1,如果Integer.MAX_VALUE,则重置为0,用于选择消息队列。
List queueDatas: topic队列元数据。
List brokerDatas: topic分布的broker元数据。
HashMap/* Filter Server */> filterServerTable: broker上过滤服务器地址列表。

第一次发送消息时,本地没有缓存topic的路由信息,查询NameServer尝试获取,如果路由信息未找到,再次尝试用默认主题DefaultMQProducerImpl#createTopicKey去查询,如果BrokerConfig#autoCreateTopicEnable为true时,NameServer将返回路由信息,如果autoCreateTopicEnable为false将抛出无法找到路由异常。代码MQClientInstance#updateTopicRouteInfoFromNameServer这个方法的功能是消息生产者更新和维护路由缓存,具体代码如下。

Step1:如果isDefault为true,则使用默认主题去查询,如果查询到路由信息,则替换路由信息中读写队列个数,为消息生产者默认的队列个数(defaultTopicQueueNums);如果isDefault为false,则使用参数topic去查询;如果未查询到路由信息,则返回false,表示路由信息未变化。

MQClientInstance#updateTopicRouteInfoFromNameServer

TopicRouteData topicRouteData;
if (isDefault && defaultMQProducer != null) {
    topicRouteData = this.mQClientAPIImpl.getDefaultTopicRouteInfoFromNameServer(defaultMQProducer.getCreateTopicKey(),
        1000 * 3);
    if (topicRouteData != null) {
        for (QueueData data : topicRouteData.getQueueDatas()) {
            int queueNums = Math.min(defaultMQProducer.getDefaultTopicQueueNums(), data.getReadQueueNums());
            data.setReadQueueNums(queueNums);
            data.setWriteQueueNums(queueNums);
        }
    }
} else {
    topicRouteData = this.mQClientAPIImpl.getTopicRouteInfoFromNameServer(topic, 1000 * 3);
}

Step2:如果路由信息未找到,与本地缓存中的路由信息进行对比,判断路由信息是否发生了改变,如果未发生变化,则直接返回false。

MQClientInstance#updateTopicRouteInfoFromNameServer

if (topicRouteData != null) {
    TopicRouteData old = this.topicRouteTable.get(topic);
    boolean changed = topicRouteDataIsChange(old, topicRouteData);
   if (!changed) {
       changed = this.isNeedUpdateTopicRouteInfo(topic);
    } else {
        log.info("the topic[{}] route info changed, old[{}] ,new[{}]", topic, old, topicRouteData);
   }

Step3:更新MQClientInstance Broker地址缓存表。

MQClientInstance#updateTopicRouteInfoFromNameServer

    // Update Pub info
    {
        TopicPublishInfo publishInfo = topicRouteData2TopicPublishInfo(topic, topicRouteData);
        publishInfo.setHaveTopicRouterInfo(true);
        Iterator> it = this.producerTable.entrySet().iterator();
        while (it.hasNext()) {
            Entry entry = it.next();
            MQProducerInner impl = entry.getValue();
            if (impl != null) {
                impl.updateTopicPublishInfo(topic, publishInfo);
            }
        }
    }

Step4: 根据topicRouteData中的List转换成topicPublishInfo的List列表。其具体实现在topicRouteData2TopicPublishInfo,然后会更新该MQClientInstance所管辖的所有消息发送关于topic的路由信息。

MQClientInstance#topicRouteData2TopicPublishInfo

    List qds = route.getQueueDatas();
    Collections.sort(qds);
    for (QueueData qd : qds) {
        if (PermName.isWriteable(qd.getPerm())) {
            BrokerData brokerData = null;
            for (BrokerData bd : route.getBrokerDatas()) {
                if (bd.getBrokerName().equals(qd.getBrokerName())) {
                    brokerData = bd;
                    break;
                }
            }

            if (null == brokerData) {
                continue;
            }

            if (!brokerData.getBrokerAddrs().containsKey(MixAll.MASTER_ID)) {
                continue;
            }

            for (int i = 0; i < qd.getWriteQueueNums(); i++) {
                MessageQueue mq = new MessageQueue(topic, qd.getBrokerName(), i);
                info.getMessageQueueList().add(mq);
            }
        }
    }

循环遍历路由信息的QueueData信息,如果队列没有写权限,则继续遍历下一个QueueData;根据brokerName找到brokerData信息,找不到或没有找到Master节点,则遍历下一个QueueData;根据写队列个数,根据topic+序号创建MessageQueue,填充topicPublishInfo的List。完成消息发送的路由查找。

选择消息队列

根据路由信息选择消息队列,返回的消息队列按照broker、序号排序。举例说明,如果topicA在broker-a,broker-b上分别创建了4个队列,那么返回的消息队列:[{"brokerName":"broker-a","queueId":0},{"brokerName":"broker-a","queueId":1},{"brokerName":"broker-a","queueId":2},{"brokerName":"broker-a","queueId":3},{"brokerName":"broker-b","queueId":0},{"brokerName":"broker-a","queueId":1},{"brokerName":"broker-a","queueId":2},{"brokerName":"broker-a","queueId":3}],那么RocketMQ如何选择消息队列呢?

首先消息发送端采用重试机制,由retryTimesWhenSendFailed指定同步方式重试次数,异步重试机制在收到消息发送结构后执行回调之前进行重试。由retryTimesWhenSendAsyncFailed指定,接下来就是循环执行,选择消息队列、发送消息,发送成功则返回,收到异常则重试。选择消息队列有两种方式。
1、sendLatencyFaultEnable = false,默认不启用Broker故障延迟机制。
2、sendLatencyFaultEnable = true,启用Broker故障延迟机制。

1.默认机制
sendLatencyFaultEnable = false,调用TopicPublishInfo#selectOneMessageQueue

TopicPublishInfo#selectOneMessageQueue

    public MessageQueue selectOneMessageQueue(final String lastBrokerName) {
        if (lastBrokerName == null) {
            return selectOneMessageQueue();
        } else {
            int index = this.sendWhichQueue.getAndIncrement();
            for (int i = 0; i < this.messageQueueList.size(); i++) {
                int pos = Math.abs(index++) % this.messageQueueList.size();
                if (pos < 0)
                    pos = 0;
                MessageQueue mq = this.messageQueueList.get(pos);
                if (!mq.getBrokerName().equals(lastBrokerName)) {
                    return mq;
                }
            }
            return selectOneMessageQueue();
        }
    }

首先在一次消息发送过程中,可能会多次执行算则消息队列这个方法,lastBrokerName就是上一次选择的执行发送消息失败的Broker。第一次执行消息队列选择时,lastBrokerName为null,此时直接用sendWhichQueue自增再获取值,与当前路由表中消息队列个数取模,返回该位置的MessageQueue(selectOneMessageQueue()方法),如果消息发送再失败的话,下次进行消息队列选择时规避上次MessageQueue所在的Broker,否则还是很有可能再次失败。

该算法在一次消息发送过程中能成功规避故障的Broker,但如果Broker宕机,由于路由算法中的消息队列是按Broker排序的,如果上一次根据路由算法选择的是宕机的Broker的第一个队列,name随后的下次选择是宕机Broker的第二个队列,消息发送很有可能会失败,再次引发重试,带来不必要的性能损耗,那么有什么方法在一次消息发送失败后,暂时将该Broker排除在消息队列选择范围外呢?或许有朋友会问,Broker不可用后,路由信息中为什么还会有包含该Broker的路由信息呢?其实这不难解释:首先,NameServer检测Broker是否可用是有延迟的,最短一次心跳检测间隔(10s);其次,NameServer不会检测到Broker当即后马上推送消息给消息生产者,而是消息生产者每个30s更新一次路由信息,所以消息生产者最快感知Broker最新的路由信息也需要30s。如果引入一种机制,在Broker宕机期间,如果一次消息发送失败后,可以将该Broker暂时排除在消息队列的选择范围中。

2.Broker故障延迟机制

MQFaultStrategy#selectOneMessageQueue

    public MessageQueue selectOneMessageQueue(final TopicPublishInfo tpInfo, final String lastBrokerName) {
        if (this.sendLatencyFaultEnable) {
            try {
                int index = tpInfo.getSendWhichQueue().getAndIncrement();
                for (int i = 0; i < tpInfo.getMessageQueueList().size(); i++) {
                    int pos = Math.abs(index++) % tpInfo.getMessageQueueList().size();
                    if (pos < 0)
                        pos = 0;
                    MessageQueue mq = tpInfo.getMessageQueueList().get(pos);
                    if (latencyFaultTolerance.isAvailable(mq.getBrokerName())) {
                        if (null == lastBrokerName || mq.getBrokerName().equals(lastBrokerName))
                            return mq;
                    }
                }

                final String notBestBroker = latencyFaultTolerance.pickOneAtLeast();
                int writeQueueNums = tpInfo.getQueueIdByBroker(notBestBroker);
                if (writeQueueNums > 0) {
                    final MessageQueue mq = tpInfo.selectOneMessageQueue();
                    if (notBestBroker != null) {
                        mq.setBrokerName(notBestBroker);
                        mq.setQueueId(tpInfo.getSendWhichQueue().getAndIncrement() % writeQueueNums);
                    }
                    return mq;
                } else {
                    latencyFaultTolerance.remove(notBestBroker);
                }
            } catch (Exception e) {
                log.error("Error occurred when selecting message queue", e);
            }

            return tpInfo.selectOneMessageQueue();
        }

        return tpInfo.selectOneMessageQueue(lastBrokerName);
    }

首先对上述代码进行解读。
1、根据对消息队列进行轮询获取一个消息队列。
2、验证该消息队列是否可用,latencyFaultTolerance.isAvailable(mq.getBrokerName())是关键。
3、如果返回的MessageQueue可用,移除lantencyFaultTolerance关于该topic条目,表明该Broker故障已经恢复。
Broker故障延迟机制核心类如下:

public interface LatencyFaultTolerance {
    void updateFaultItem(final T name, final long currentLatency, final long notAvailableDuration);
    boolean isAvailable(final T name);
    void remove(final T name);
    T pickOneAtLeast();
}

public class LatencyFaultToleranceImpl implements LatencyFaultTolerance {
    private final ConcurrentHashMap faultItemTable = new ConcurrentHashMap(16);
    private final ThreadLocalIndex whichItemWorst = new ThreadLocalIndex();
}

class FaultItem implements Comparable {
    private final String name;
    private volatile long currentLatency;
    private volatile long startTimestamp;
}

public class MQFaultStrategy {
    private final LatencyFaultTolerance latencyFaultTolerance = new LatencyFaultToleranceImpl();
    private boolean sendLatencyFaultEnable = false;
    private long[] latencyMax = {50L, 100L, 550L, 1000L, 2000L, 3000L, 15000L};
    private long[] notAvailableDuration = {0L, 0L, 30000L, 60000L, 120000L, 180000L, 600000L};
}

LatencyFaultTolerance: 延迟机制接口规范。
1、

void updateFaultItem(final T name, final long currentLatency, final long notAvailableDuration);更新失败条目。
name: brokerName
currentLatenmcy: 消息发送故障延迟时间。
notAvailableDuration: 不可用持续时长,在这个时间内,Broker将被规避。
2、

boolean isAvailable(final T name);

判断Broker是否可用。
name: broker名称。
3、

void remove(final T name)
移除Fault条目,意味着Broker重新参与路由计算。
4、

T pickOneAtLeast() 尝试从规避的Broker中选择一个可用的Broker,如果没有找到,将返回null。
FaultItem: 失败条目(规避规则条目)。
final String name 条目唯一键,这里为brokerName。
private volatile long currentLatency 本次消息发送延迟。
private volatile long startTimeStamp 故障规避开始时间。
MQFaultStrategy: 消息失败策略,延迟实现的门面类。
long[] latencyMax = {50L, 100L, 550L, 1000L, 2000L, 3000L, 15000L};
long[] notAvailableDuration = {0L, 0L, 30000L, 60000L, 120000L, 180000L, 600000L};
latencyMax,根据currentLatency本次消息发送延迟,从latencyMax尾部向前汇总爱到第一个比currentLatency小的索引index,如果没有找到,返回0。然后根据这个索引从notAvailableDuration数组中取出对应的时间,在这个时长内,Broker将设置为不可用。

DefaultMQProducerImpl#sendDefaultImpl

beginTimestampPrev = System.currentTimeMillis();
sendResult = this.sendKernelImpl(msg, mq, communicationMode, sendCallback, topicPublishInfo, timeout);
endTimestamp = System.currentTimeMillis();
this.updateFaultItem(mq.getBrokerName(), endTimestamp - beginTimestampPrev, false);

上述代码如果发送过程中抛出了异常,调用DefaultMQProducerImpl#updateFaultItem,该方法则直接调用MQFaultStrategy#updateFaultItem方法,关注一下各个参数的含义。
第一个参数: broker名称。
第二个参数: 本次消息发送延迟时间 currentLatency。
第三个参数: isolation,是否隔离,该参数的含义如果为true,则使用默认时长30s来计算Broker故障规避时长,如果为false,则使用本次消息发送延迟时间来计算Broker故障规避时长。

MQFaultStrategy#updateFaultItem

    public void updateFaultItem(final String brokerName, final long currentLatency, boolean isolation) {
        if (this.sendLatencyFaultEnable) {
            long duration = computeNotAvailableDuration(isolation ? 30000 : currentLatency);
            this.latencyFaultTolerance.updateFaultItem(brokerName, currentLatency, duration);
        }
    }

    private long computeNotAvailableDuration(final long currentLatency) {
        for (int i = latencyMax.length - 1; i >= 0; i--) {
            if (currentLatency >= latencyMax[i])
                return this.notAvailableDuration[i];
        }

        return 0;
    }

如果isolation为true,则使用30s作为computeNotAvailableDuration方法的参数;如果isolation为false,则使用本次消息发送时延作为computeNotAvailableDuration方法的参数,那computeNotAvailableDuration的作用是计算因本次消息发送故障需要将Broker规避的时长,也就是接下来多久的时间内该Broker将不参与消息发送队列负载。具体算法: 从latencyMax数组尾部开始寻找,找到第一个比currentLatency小的下标,然后从notAvailableDuration数组中获取需要规避的时长,该方法最终调用LatencyFaultTolerance的updateFaultItem。

LatencyFaultToleranceImpl#updateFaultItem

public void updateFaultItem(final String name, final long currentLatency, final long notAvailableDuration) {
    FaultItem old = this.faultItemTable.get(name);
    if (null == old) {
        final FaultItem faultItem = new FaultItem(name);
        faultItem.setCurrentLatency(currentLatency);
        faultItem.setStartTimestamp(System.currentTimeMillis() + notAvailableDuration);

        old = this.faultItemTable.putIfAbsent(name, faultItem);
        if (old != null) {
            old.setCurrentLatency(currentLatency);
            old.setStartTimestamp(System.currentTimeMillis() + notAvailableDuration);
        }
    } else {
        old.setCurrentLatency(currentLatency);
        old.setStartTimestamp(System.currentTimeMillis() + notAvailableDuration);
    }
}

根据broker名称从缓存表中获取FaultItem,如果找到则更新FaultItem,否则创建FaultItem。这里有两个关键点。
1、currentLatency、startTimeStamp被volatile修饰。
2、startTimeStamp为当前系统时间加上需要规避的时长。startTimeStamp是判断broker当前是否可用的直接依据,请看FaultItem#isAvailable方法。

public boolean isAvailable() {
    return (System.currentTimeMillis() - startTimestamp) >= 0;
}

消息发送

消息发送API核心入口: DefaultMQProducerImpl#sendKernelImpl

private SendResult sendKernelImpl(final Message msg,
    final MessageQueue mq,
    final CommunicationMode communicationMode,
    final SendCallback sendCallback,
    final TopicPublishInfo topicPublishInfo,
    final long timeout)

消息发送参数详解。
1、Message msg: 待发送消息。
2、MessageQueue mq: 消息将发送到该消息队列上。
3、CommunicationMode communicationMode: 消息发送模式,SYNC、ASYNC、ONEWAY。
4、SendCallback sendCallback: 异步消息回调函数。
5、TopicPublishInfo topicPublishInfo: 主题路由信息。
6、long timeout: 消息发送超时时间。

Step1: 根据MessageQueue获取Broker的网络地址。如果MQClientInstance的brokerAddrTable未缓存该Broker的信息,则从NameServer主动更新一下topic的路由信息。如果路由更新后还是找不到Broker信息,则抛出MQClientException,提示Broker不存在。