源码解析-RocketMQ-broker

1.前言

Broker是整个RocketMQ的业务核心，所有消息存储、转发这些最为重要的业务都是在Broker中进行处理的。而Broker的内部架构，有点类似于JavaWeb开发的MVC架构。有Controller负责响应请求，各种Service组件负责具体业务，然后还有负责消息存盘的功能模块则类似于Dao。

2.源码分析

2.1 启动类BrokerStartup

public static void main(String[] args) {
        start(createBrokerController(args));
}

	//创建Broker核心配置
    public static BrokerController createBrokerController(String[] args) {
     
        try {
            //Broker的核心配置信息
            final BrokerConfig brokerConfig = new BrokerConfig();
            final NettyServerConfig nettyServerConfig = new NettyServerConfig();
            final NettyClientConfig nettyClientConfig = new NettyClientConfig();
 
            //Netty服务端的监听端口10911
            nettyServerConfig.setListenPort(10911);
            //Broker用来存储消息的一些配置信息。
            final MessageStoreConfig messageStoreConfig = new MessageStoreConfig();
            //如果是SLAVE，会设置一个参数。这参数干嘛的，可以去官网查查。
            if (BrokerRole.SLAVE == messageStoreConfig.getBrokerRole()) {
                int ratio = messageStoreConfig.getAccessMessageInMemoryMaxRatio() - 10;
                messageStoreConfig.setAccessMessageInMemoryMaxRatio(ratio);
            }
      
            //判断集群角色。通过BrokerId判断主从
            switch (messageStoreConfig.getBrokerRole()) {
                case ASYNC_MASTER:
                case SYNC_MASTER:
                    brokerConfig.setBrokerId(MixAll.MASTER_ID);
                    break;
                case SLAVE:
                    if (brokerConfig.getBrokerId() <= 0) {
                        System.out.printf("Slave's brokerId must be > 0");
                        System.exit(-3);
                    }

                    break;
                default:
                    break;
            }
            /判断是否基于Dledger技术来管理主从同步和CommitLog的条件就是brokerId设置为-1
            if (messageStoreConfig.isEnableDLegerCommitLog()) {
                brokerConfig.setBrokerId(-1);
            }
			
			//消息持久化监听端口10912
            messageStoreConfig.setHaListenPort(nettyServerConfig.getListenPort() + 1);
        
            //创建Controllerg
            final BrokerController controller = new BrokerController(
                brokerConfig,
                nettyServerConfig,
                nettyClientConfig,
                messageStoreConfig);
            // remember all configs to prevent discard
            controller.getConfiguration().registerConfig(properties);
            //初始化 注意从中理清楚Broker的组件结构
            boolean initResult = controller.initialize();
            if (!initResult) {
                controller.shutdown();
                System.exit(-3);
            }
        return null;
    }

这里类似于namesrv的流程，createBrokerController(args)；代码前部分大多都是创建broker一些核心配置等信息。

final BrokerConfig brokerConfig = new BrokerConfig();//broker配置
final NettyServerConfig nettyServerConfig = new NettyServerConfig();//netty服务端配置，Netty服务端占用了10911端口。同样也可以在配置文件中覆盖。
final NettyClientConfig nettyClientConfig = new NettyClientConfig();//netty客户端配置
final MessageStoreConfig messageStoreConfig = new MessageStoreConfig();//Broker用来存储消息的一些配置信息。

这里说明broker既可以作为服务端提供服务（提供消息的存储等服务），也可以作为客户端发起请求（事物消息请求生产者或者向namesrv发起注册请求等）。

2.2 controller.initializ() 初始化

源码位置在BrokerController#initialize();初始化方法initialize()主要分一下几个步骤：

2.2.1 加载磁盘上的配置信息

//加载磁盘上的配置信息。这些配置信息就用到了MessageStoreConfig
1⃣️boolean result = this.topicConfigManager.load();
2⃣️result = result && this.consumerOffsetManager.load();
3⃣️result = result && this.subscriptionGroupManager.load();
4⃣️result = result && this.consumerFilterManager.load();

跟踪以上源码可以分别发现
首先会加载config配置文件。一般默认位置在根目录/store/config下，比如我的在/Users/weiqiang/app/rocketmq/store/config

这些文件每个都有两个，一个是正常的，一个是备份；加载顺序不用多说了，肯定是先加载正常的。加载文件会进行decode()；这里源码不同的xxxManager对应不同的decode()方法实现（模版设计模式，子类实现具体细节）；
1⃣️2⃣️3⃣️4⃣️分别加载对应的文件如下

    public static String getTopicConfigPath(final String rootDir) {
        return rootDir + File.separator + "config" + File.separator + "topics.json";
    }

    public static String getConsumerOffsetPath(final String rootDir) {
        return rootDir + File.separator + "config" + File.separator + "consumerOffset.json";
    }

    public static String getSubscriptionGroupPath(final String rootDir) {
        return rootDir + File.separator + "config" + File.separator + "subscriptionGroup.json";
    }

    public static String getConsumerFilterPath(final String rootDir) {
        return rootDir + File.separator + "config" + File.separator + "consumerFilter.json";
    }

2.2.2 初始化消息，存储load()的内容

2.2.1我们刚刚加载完的内容要干嘛，这里我们就是要进行消息初始化到内存中去。

if (result) {
    try {
        //消息存储管理组件，管理磁盘上的消息的。
        this.messageStore =
            new DefaultMessageStore(this.messageStoreConfig, this.brokerStatsManager, this.messageArrivingListener,
                this.brokerConfig);
        //如果启用了Dledger，他就初始化一堆Dledger相关的组件
        if (messageStoreConfig.isEnableDLegerCommitLog()) {
            DLedgerRoleChangeHandler roleChangeHandler = new DLedgerRoleChangeHandler(this, (DefaultMessageStore) messageStore);
            ((DLedgerCommitLog)((DefaultMessageStore) messageStore).getCommitLog()).getdLedgerServer().getdLedgerLeaderElector().addRoleChangeHandler(roleChangeHandler);
        }
        //broker的统计组件
        this.brokerStats = new BrokerStats((DefaultMessageStore) this.messageStore);
        //load plugin
        //加载插件？
        MessageStorePluginContext context = new MessageStorePluginContext(messageStoreConfig, brokerStatsManager, messageArrivingListener, brokerConfig);
        this.messageStore = MessageStoreFactory.build(context, this.messageStore);
        this.messageStore.getDispatcherList().addFirst(new CommitLogDispatcherCalcBitMap(this.brokerConfig, this.consumerFilterManager));
    } catch (IOException e) {
        result = false;
        log.error("Failed to initialize", e);
    }
 }

这里不做详细介绍，简单梳理下流程，后续会进行消息存储这块的介绍。

2.3 初始化线程池&和注册绑定各种处理器

//发送消息的处理线程池
this.sendMessageExecutor = new BrokerFixedThreadPoolExecutor(
    this.brokerConfig.getSendMessageThreadPoolNums(),
    this.brokerConfig.getSendMessageThreadPoolNums(),
    1000 * 60,
    TimeUnit.MILLISECONDS,
    this.sendThreadPoolQueue,
    new ThreadFactoryImpl("SendMessageThread_"));
//处理consumer的pull请求的线程池
this.pullMessageExecutor = new BrokerFixedThreadPoolExecutor(
    this.brokerConfig.getPullMessageThreadPoolNums(),
    this.brokerConfig.getPullMessageThreadPoolNums(),
    1000 * 60,
    TimeUnit.MILLISECONDS,
    this.pullThreadPoolQueue,
    new ThreadFactoryImpl("PullMessageThread_"));
//回复消息的线程池
this.replyMessageExecutor = new BrokerFixedThreadPoolExecutor(
    this.brokerConfig.getProcessReplyMessageThreadPoolNums(),
    this.brokerConfig.getProcessReplyMessageThreadPoolNums(),
    1000 * 60,
    TimeUnit.MILLISECONDS,
    this.replyThreadPoolQueue,
    new ThreadFactoryImpl("ProcessReplyMessageThread_"));
//查询消息的线程池。
this.queryMessageExecutor = new BrokerFixedThreadPoolExecutor(
    this.brokerConfig.getQueryMessageThreadPoolNums(),
    this.brokerConfig.getQueryMessageThreadPoolNums(),
    1000 * 60,
    TimeUnit.MILLISECONDS,
    this.queryThreadPoolQueue,
    new ThreadFactoryImpl("QueryMessageThread_"));
//这个一看就是管理Broker的一些命令执行的线程池
this.adminBrokerExecutor =
    Executors.newFixedThreadPool(this.brokerConfig.getAdminBrokerThreadPoolNums(), new ThreadFactoryImpl(
        "AdminBrokerThread_"));
//这个就是管理客户端的线程池
this.clientManageExecutor = new ThreadPoolExecutor(
    this.brokerConfig.getClientManageThreadPoolNums(),
    this.brokerConfig.getClientManageThreadPoolNums(),
    1000 * 60,
    TimeUnit.MILLISECONDS,
    this.clientManagerThreadPoolQueue,
    new ThreadFactoryImpl("ClientManageThread_"));
//心跳请求线程池
this.heartbeatExecutor = new BrokerFixedThreadPoolExecutor(
    this.brokerConfig.getHeartbeatThreadPoolNums(),
    this.brokerConfig.getHeartbeatThreadPoolNums(),
    1000 * 60,
    TimeUnit.MILLISECONDS,
    this.heartbeatThreadPoolQueue,
    new ThreadFactoryImpl("HeartbeatThread_", true));
//事务消息结束线程池？一看就是跟事务消息有关。
this.endTransactionExecutor = new BrokerFixedThreadPoolExecutor(
    this.brokerConfig.getEndTransactionThreadPoolNums(),
    this.brokerConfig.getEndTransactionThreadPoolNums(),
    1000 * 60,
    TimeUnit.MILLISECONDS,
    this.endTransactionThreadPoolQueue,
    new ThreadFactoryImpl("EndTransactionThread_"));
//管理consumer的线程池
this.consumerManageExecutor =
    Executors.newFixedThreadPool(this.brokerConfig.getConsumerManageThreadPoolNums(), new ThreadFactoryImpl(
        "ConsumerManageThread_"));

//注册各种处理器。用上了之前那一大堆的线程池。
this.registerProcessor();在这里进行线程池跟处理的绑定，例如

this.remotingServer.registerProcessor(RequestCode.SEND_MESSAGE, sendProcessor, this.sendMessageExecutor);

2.4 初始化定时任务

• 初始化broker统计任务
• 初始化consumer消费offset持久化到磁盘任务
• 初始化对consumer的filter过滤器进行持久化任务
• 初始化落后commitlog分发任务

到这里,createBrokerController()就分析完了。总之，现在所有的准备条件，都已经就位。

3 controller.start()

BrokerController创建完了之后，start就比较简单了，把之前创建的各种服务都start起来。同时向所有的Namesrv注册自己。
启动定时器，定时向所有的namesrv注册自己，维持心跳链接，告诉namesrv自己还活着，防止namesrv剔除自己，这样消息生产者和消费者就可以通过namesrv找到自己。

3.1 broker发送心跳请求

public synchronized void registerBrokerAll(final boolean checkOrderConfig, boolean oneway, boolean forceRegister) {
        //Topic配置相关的东西
        TopicConfigSerializeWrapper topicConfigWrapper = this.getTopicConfigManager().buildTopicConfigSerializeWrapper();

         if (!PermName.isWriteable(this.getBrokerConfig().getBrokerPermission())
            || !PermName.isReadable(this.getBrokerConfig().getBrokerPermission())) {
            ConcurrentHashMap topicConfigTable = new ConcurrentHashMap();
            for (TopicConfig topicConfig : topicConfigWrapper.getTopicConfigTable().values()) {
                TopicConfig tmp =
                    new TopicConfig(topicConfig.getTopicName(), topicConfig.getReadQueueNums(), topicConfig.getWriteQueueNums(),
                        this.brokerConfig.getBrokerPermission());
                topicConfigTable.put(topicConfig.getTopicName(), tmp);
            }
            topicConfigWrapper.setTopicConfigTable(topicConfigTable);
        }
        //这里才是比较关键的地方。先判断是否需要注册，然后调用doRegisterBrokerAll方法真正去注册。
        if (forceRegister || needRegister(this.brokerConfig.getBrokerClusterName(),
            this.getBrokerAddr(),
            this.brokerConfig.getBrokerName(),
            this.brokerConfig.getBrokerId(),
            this.brokerConfig.getRegisterBrokerTimeoutMills())) {
            doRegisterBrokerAll(checkOrderConfig, oneway, topicConfigWrapper);
        }
    }

一路跟踪源码到Broker真正执行注册的方法

RegisterBrokerResult result = registerBroker(namesrvAddr,oneway, timeoutMills,requestHeader,body);


private RegisterBrokerResult registerBroker(
        final String namesrvAddr,
        final boolean oneway,
        final int timeoutMills,
        final RegisterBrokerRequestHeader requestHeader,
        final byte[] body
    ) throws RemotingCommandException, MQBrokerException, RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException,
        InterruptedException {
        //封装一个网络请求
        RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.REGISTER_BROKER, requestHeader);
        request.setBody(body);
        //特殊请求 sendOneWay
        if (oneway) {
            try {
                this.remotingClient.invokeOneway(namesrvAddr, request, timeoutMills);
            } catch (RemotingTooMuchRequestException e) {
                // Ignore
            }
            return null;
        }
        //真正发送网络请求的地方。这个remotingClient就是一个NettyClient。
        RemotingCommand response = this.remotingClient.invokeSync(namesrvAddr, request, timeoutMills);
        //封装网络请求结果。
        assert response != null;
        switch (response.getCode()) {
            case ResponseCode.SUCCESS: {
                RegisterBrokerResponseHeader responseHeader =
                    (RegisterBrokerResponseHeader) response.decodeCommandCustomHeader(RegisterBrokerResponseHeader.class);
                RegisterBrokerResult result = new RegisterBrokerResult();
                result.setMasterAddr(responseHeader.getMasterAddr());
                result.setHaServerAddr(responseHeader.getHaServerAddr());
                if (response.getBody() != null) {
                    result.setKvTable(KVTable.decode(response.getBody(), KVTable.class));
                }
                return result;
            }
            default:
                break;
        }
        //请求不成功就直接抛出异常。
        throw new MQBrokerException(response.getCode(), response.getRemark());
    }

这里我们发现borker和NameServer之间通过netty进行网络传输，Broker向NameServer发起注册时会在请求中添加注册码RequestCode.REGISTER_BROKER。RocketMQ中的每个请求都会定义一个requestCode，服务端的网络处理器会根据不同的requestCode进行影响的业务处理。