RocketMQ二、producer端启动
前面简单了解了一下RocketMQ的使用。本文分析一下producer端的启动流程。
首先还是看一下RocketMQ的配置类：
@Bean
@ConditionalOnMissingBean(DefaultMQProducer.class)
@ConditionalOnProperty(prefix = "rocketmq", value = {"name-server", "producer.group"})
public DefaultMQProducer defaultMQProducer(RocketMQProperties rocketMQProperties) {
RocketMQProperties.Producer producerConfig = rocketMQProperties.getProducer();
String nameServer = rocketMQProperties.getNameServer();
String groupName = producerConfig.getGroup();
Assert.hasText(nameServer, "[rocketmq.name-server] must not be null");
Assert.hasText(groupName, "[rocketmq.producer.group] must not be null");
DefaultMQProducer producer;
String ak = rocketMQProperties.getProducer().getAccessKey();
String sk = rocketMQProperties.getProducer().getSecretKey();
if (!StringUtils.isEmpty(ak) && !StringUtils.isEmpty(sk)) {
producer = new DefaultMQProducer(groupName, new AclClientRPCHook(new SessionCredentials(ak, sk)),
rocketMQProperties.getProducer().isEnableMsgTrace(),
rocketMQProperties.getProducer().getCustomizedTraceTopic());
producer.setVipChannelEnabled(false);
} else {
producer = new DefaultMQProducer(groupName, rocketMQProperties.getProducer().isEnableMsgTrace(),
rocketMQProperties.getProducer().getCustomizedTraceTopic());
}
producer.setNamesrvAddr(nameServer);
producer.setSendMsgTimeout(producerConfig.getSendMessageTimeout());
producer.setRetryTimesWhenSendFailed(producerConfig.getRetryTimesWhenSendFailed());
producer.setRetryTimesWhenSendAsyncFailed(producerConfig.getRetryTimesWhenSendAsyncFailed());
producer.setMaxMessageSize(producerConfig.getMaxMessageSize());
producer.setCompressMsgBodyOverHowmuch(producerConfig.getCompressMessageBodyThreshold());
producer.setRetryAnotherBrokerWhenNotStoreOK(producerConfig.isRetryNextServer());
return producer;
}
@Bean(destroyMethod = "destroy")
@ConditionalOnBean(DefaultMQProducer.class)
@ConditionalOnMissingBean(RocketMQTemplate.class)
public RocketMQTemplate rocketMQTemplate(DefaultMQProducer mqProducer, ObjectMapper rocketMQMessageObjectMapper) {
RocketMQTemplate rocketMQTemplate = new RocketMQTemplate();
rocketMQTemplate.setProducer(mqProducer);
rocketMQTemplate.setObjectMapper(rocketMQMessageObjectMapper);
return rocketMQTemplate;
}
主要定义了两个bean：DefaultMQProducer和RocketMQTemplate
先看一下DefaultMQProducer的初始化：
public DefaultMQProducer(final String producerGroup, RPCHook rpcHook, boolean enableMsgTrace,final String customizedTraceTopic) {
this.producerGroup = producerGroup;
defaultMQProducerImpl = new DefaultMQProducerImpl(this, rpcHook);
//if client open the message trace feature
if (enableMsgTrace) {
try {
AsyncTraceDispatcher dispatcher = new AsyncTraceDispatcher(customizedTraceTopic, rpcHook);
dispatcher.setHostProducer(this.getDefaultMQProducerImpl());
traceDispatcher = dispatcher;
this.getDefaultMQProducerImpl().registerSendMessageHook(
new SendMessageTraceHookImpl(traceDispatcher));
} catch (Throwable e) {
log.error("system mqtrace hook init failed ,maybe can't send msg trace data");
}
}
}
可以看到主要是初始化了一个DefaultMQProducerImpl实例，如果设置了消息跟踪，则设置跟踪的分发器AsyncTraceDispatcher。
看一下DefaultMQProducerImpl的初始化：
public DefaultMQProducerImpl(final DefaultMQProducer defaultMQProducer, RPCHook rpcHook) {
this.defaultMQProducer = defaultMQProducer;
this.rpcHook = rpcHook;
this.asyncSenderThreadPoolQueue = new LinkedBlockingQueue(50000);
this.defaultAsyncSenderExecutor = new ThreadPoolExecutor(
Runtime.getRuntime().availableProcessors(),
Runtime.getRuntime().availableProcessors(),
1000 * 60,
TimeUnit.MILLISECONDS,
this.asyncSenderThreadPoolQueue,
new ThreadFactory() {
private AtomicInteger threadIndex = new AtomicInteger(0);
@Override
public Thread newThread(Runnable r) {
return new Thread(r, "AsyncSenderExecutor_" + this.threadIndex.incrementAndGet());
}
});
}
也没做什么，主要是初始化了一个异步发送消息用的线程池。
回过头来想一下，当我们引入RocketMQ时，至少需要配置一个nameserver的服务地址。那么也就是说我们的服务启动时至少要有一个和nameserver通信的过程。这个操作在哪儿呢？
我们看一下RocketMQTemplate的定义：
public class RocketMQTemplate extends AbstractMessageSendingTemplate implements InitializingBean, DisposableBean {
可以看到这里实现了InitializingBean接口，也就是在实例化template并设置完属性后，会调用afterPropertiesSet方法：
@Override
public void afterPropertiesSet() throws Exception {
Assert.notNull(producer, "Property 'producer' is required");
producer.start();
}
好的，可以看到调用了producer的start方法：
@Override
public void start() throws MQClientException {
this.defaultMQProducerImpl.start();
if (null != traceDispatcher) {
try {
traceDispatcher.start(this.getNamesrvAddr());
} catch (MQClientException e) {
log.warn("trace dispatcher start failed ", e);
}
}
}
我们主要关注producer本身的start，这里关于消息跟踪分发器的start暂时先不看。
public void start(final boolean startFactory) throws MQClientException {
//判断producer当前的状态, 初始化完成后是CREATE_JUST状态
switch (this.serviceState) {
//初始化完成
case CREATE_JUST:
this.serviceState = ServiceState.START_FAILED;
//校验producerGroupName是否合法, 同一个进程内, producerGroupName必须是唯一的.
this.checkConfig();
//用进程id作为producerGroup的默认值
if (!this.defaultMQProducer.getProducerGroup().equals(MixAll.CLIENT_INNER_PRODUCER_GROUP)) {
this.defaultMQProducer.changeInstanceNameToPID();
}
//mQClientFactory主要负责与nameserve和broker的通信,
this.mQClientFactory = MQClientManager.getInstance().getOrCreateMQClientInstance(this.defaultMQProducer, rpcHook);
//将produerGroup, produer注册到本地的producerTable表中, 一个produerGroup对应一个producer
boolean registerOK = mQClientFactory.registerProducer(this.defaultMQProducer.getProducerGroup(), this);
if (!registerOK) {
this.serviceState = ServiceState.CREATE_JUST;
throw new MQClientException("The producer group[" + this.defaultMQProducer.getProducerGroup()
+ "] has been created before, specify another name please." + FAQUrl.suggestTodo(FAQUrl.GROUP_NAME_DUPLICATE_URL),
null);
}
//在topicPublishInfoTable表中, 放入一个默认的topic和它的路由信息
this.topicPublishInfoTable.put(this.defaultMQProducer.getCreateTopicKey(), new TopicPublishInfo());
if (startFactory) {
//启动mQClientFactory
mQClientFactory.start();
}
log.info("the producer [{}] start OK. sendMessageWithVIPChannel={}", this.defaultMQProducer.getProducerGroup(),
this.defaultMQProducer.isSendMessageWithVIPChannel());
//serviceState状态变更为RUNNING
this.serviceState = ServiceState.RUNNING;
break;
case RUNNING:
case START_FAILED:
case SHUTDOWN_ALREADY:
throw new MQClientException("The producer service state not OK, maybe started once, "
+ this.serviceState
+ FAQUrl.suggestTodo(FAQUrl.CLIENT_SERVICE_NOT_OK),
null);
default:
break;
}
//发送心跳给所有的broker
this.mQClientFactory.sendHeartbeatToAllBrokerWithLock();
// 处理过期请求
this.timer.scheduleAtFixedRate(new TimerTask() {
@Override
public void run() {
try {
RequestFutureTable.scanExpiredRequest();
} catch (Throwable e) {
log.error("scan RequestFutureTable exception", e);
}
}
}, 1000 * 3, 1000);
}
可以看到一个状态验证的switch，实际上只有CREATE_JUST（producer刚实例化完成）才需要启动各种内容，而其他状态是非法状态，直接抛异常出去。
这里我们主要关注mQClientFactory，它是用来和nameserver、broker通信的客户端工厂。我们看一下它的构建，
MQClientManager的getOrCreateMQClientInstance方法：
public MQClientInstance getOrCreateMQClientInstance(final ClientConfig clientConfig, RPCHook rpcHook) {
String clientId = clientConfig.buildMQClientId();
MQClientInstance instance = this.factoryTable.get(clientId);
if (null == instance) {
instance =
new MQClientInstance(clientConfig.cloneClientConfig(),
this.factoryIndexGenerator.getAndIncrement(), clientId, rpcHook);
MQClientInstance prev = this.factoryTable.putIfAbsent(clientId, instance);
if (prev != null) {
instance = prev;
log.warn("Returned Previous MQClientInstance for clientId:[{}]", clientId);
} else {
log.info("Created new MQClientInstance for clientId:[{}]", clientId);
}
}
return instance;
}
factoryTable维护了一个map，保存clientId和mQClientFactory的关系，也就是说每个mQClientFactory对应一个客户端。这个clientId类似：IP@PID格式，唯一标识一个客户端（还和unitName有关，这里就不扩展了，一般我们一个进程只会有一个clientId，也就对应一个mQClientFactory）。
下面看一下mQClientFactory的start方法：
public void start() throws MQClientException {
synchronized (this) {
switch (this.serviceState) {
case CREATE_JUST:
this.serviceState = ServiceState.START_FAILED;
// If not specified,looking address from name server
if (null == this.clientConfig.getNamesrvAddr()) {
this.mQClientAPIImpl.fetchNameServerAddr();
}
// Start request-response channel
this.mQClientAPIImpl.start();
// Start various schedule tasks
this.startScheduledTask();
// Start pull service
this.pullMessageService.start();
// Start rebalance service
this.rebalanceService.start();
// Start push service
this.defaultMQProducer.getDefaultMQProducerImpl().start(false);
log.info("the client factory [{}] start OK", this.clientId);
this.serviceState = ServiceState.RUNNING;
break;
case START_FAILED:
throw new MQClientException("The Factory object[" + this.getClientId() + "] has been created before, and failed.", null);
default:
break;
}
}
}
首先是状态的二次校验，同样是CREATE_JUST时才会有相应操作。5步操作，下面一个个分析：
-
this.mQClientAPIImpl.fetchNameServerAddr();
public String fetchNameServerAddr() { try { String addrs = this.topAddressing.fetchNSAddr(); if (addrs != null) { if (!addrs.equals(this.nameSrvAddr)) { log.info("name server address changed, old=" + this.nameSrvAddr + ", new=" + addrs); this.updateNameServerAddressList(addrs); this.nameSrvAddr = addrs; return nameSrvAddr; } } } catch (Exception e) { log.error("fetchNameServerAddr Exception", e); } return nameSrvAddr; }简单说明一下，这里和nameserver的动态发现有关，当发现nameserver的地址有变更时，同步调整相应对象中的地址信息。这里有个topAddressing，通过http请求获取最新的nameserver数据。 对应的请求地址获取方法MixAll.getWSAddr：
public static String getWSAddr() { String wsDomainName = System.getProperty("rocketmq.namesrv.domain", DEFAULT_NAMESRV_ADDR_LOOKUP); String wsDomainSubgroup = System.getProperty("rocketmq.namesrv.domain.subgroup", "nsaddr"); String wsAddr = "http://" + wsDomainName + ":8080/rocketmq/" + wsDomainSubgroup; if (wsDomainName.indexOf(":") > 0) { wsAddr = "http://" + wsDomainName + "/rocketmq/" + wsDomainSubgroup; } return wsAddr; }和两个系统参数有关
-
this.mQClientAPIImpl.start();
跟到RemotingService的start方法，这里要明确的一点，不管是producer端还是consumer端，对于RocketMQ来说都是client端，也就是说这里我们看NettyRemotingClient实现：
@Override public void start() { //构建一个DefaultEventExecutorGroup, 用于处理netty handler中的操作 this.defaultEventExecutorGroup = new DefaultEventExecutorGroup( //客户端线程数 默认4个 nettyClientConfig.getClientWorkerThreads(), new ThreadFactory() { private AtomicInteger threadIndex = new AtomicInteger(0); @Override public Thread newThread(Runnable r) { return new Thread(r, "NettyClientWorkerThread_" + this.threadIndex.incrementAndGet()); } }); //初始化netty Bootstrap handler = this.bootstrap.group(this.eventLoopGroupWorker).channel(NioSocketChannel.class) .option(ChannelOption.TCP_NODELAY, true) .option(ChannelOption.SO_KEEPALIVE, false) .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, nettyClientConfig.getConnectTimeoutMillis()) .option(ChannelOption.SO_SNDBUF, nettyClientConfig.getClientSocketSndBufSize()) .option(ChannelOption.SO_RCVBUF, nettyClientConfig.getClientSocketRcvBufSize()) .handler(new ChannelInitializer() { @Override public void initChannel(SocketChannel ch) throws Exception { ChannelPipeline pipeline = ch.pipeline(); if (nettyClientConfig.isUseTLS()) { if (null != sslContext) { pipeline.addFirst(defaultEventExecutorGroup, "sslHandler", sslContext.newHandler(ch.alloc())); log.info("Prepend SSL handler"); } else { log.warn("Connections are insecure as SSLContext is null!"); } } pipeline.addLast( defaultEventExecutorGroup, //编码handler new NettyEncoder(), //解码handler new NettyDecoder(), //心跳检测 new IdleStateHandler(0, 0, nettyClientConfig.getClientChannelMaxIdleTimeSeconds()), //连接管理handler,处理connect, disconnect, close等事件 new NettyConnectManageHandler(), //处理接收到RemotingCommand消息后的事件, 收到服务器端响应后的相关操作 new NettyClientHandler()); } }); //定时扫描responseTable,获取返回结果,并且处理超时 延迟3秒 1秒周期 this.timer.scheduleAtFixedRate(new TimerTask() { @Override public void run() { try { NettyRemotingClient.this.scanResponseTable(); } catch (Throwable e) { log.error("scanResponseTable exception", e); } } }, 1000 * 3, 1000); if (this.channelEventListener != null) { this.nettyEventExecutor.start(); } } 可以看到netty的bootstrap的初始化了，注意这里并没有建立链接。实际上对于mq来说，客户端需要和哪个broker建立链接是有负载规则的，不同的topic会链接到不同的broker上， 也就是说真正建立链接是在某个消息第一次发送时。 这部分后面再分析。
-
this.startScheduledTask();
private void startScheduledTask() { if (null == this.clientConfig.getNamesrvAddr()) { this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() { @Override public void run() { try { MQClientInstance.this.mQClientAPIImpl.fetchNameServerAddr(); } catch (Exception e) { log.error("ScheduledTask fetchNameServerAddr exception", e); } } }, 1000 * 10, 1000 * 60 * 2, TimeUnit.MILLISECONDS); } this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() { @Override public void run() { try { MQClientInstance.this.updateTopicRouteInfoFromNameServer(); } catch (Exception e) { log.error("ScheduledTask updateTopicRouteInfoFromNameServer exception", e); } } }, 10, this.clientConfig.getPollNameServerInterval(), TimeUnit.MILLISECONDS); this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() { @Override public void run() { try { MQClientInstance.this.cleanOfflineBroker(); MQClientInstance.this.sendHeartbeatToAllBrokerWithLock(); } catch (Exception e) { log.error("ScheduledTask sendHeartbeatToAllBroker exception", e); } } }, 1000, this.clientConfig.getHeartbeatBrokerInterval(), TimeUnit.MILLISECONDS); this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() { @Override public void run() { try { MQClientInstance.this.persistAllConsumerOffset(); } catch (Exception e) { log.error("ScheduledTask persistAllConsumerOffset exception", e); } } }, 1000 * 10, this.clientConfig.getPersistConsumerOffsetInterval(), TimeUnit.MILLISECONDS); this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() { @Override public void run() { try { MQClientInstance.this.adjustThreadPool(); } catch (Exception e) { log.error("ScheduledTask adjustThreadPool exception", e); } } }, 1, 1, TimeUnit.MINUTES); }这里有好几个任务：
- 10秒延迟2分钟周期的nameserver更新任务
- 10毫秒延迟30秒周期的topic路由规则更新任务
- 1秒延迟30秒周期的broker心跳任务（也会清理掉没有心跳响应的离线broker）
- 10秒延迟5秒周期的持久化consumer的offset的任务（本地持久化/提交给broker）
- 1分钟延迟1分钟周期的调整线程池参数的任务（调整消息发送线程池核心线程数）
-
this.pullMessageService.start();
启动拉消息的线程PullMessageService.run：
@Override
public void run() {
log.info(this.getServiceName() + " service started");
while (!this.isStopped()) {
try {
PullRequest pullRequest = this.pullRequestQueue.take();
this.pullMessage(pullRequest);
} catch (InterruptedException ignored) {
} catch (Exception e) {
log.error("Pull Message Service Run Method exception", e);
}
}
log.info(this.getServiceName() + " service end");
}
从队列中take消息，然后执行pullMessage方法
-
this.rebalanceService.start();
更新负载RebalanceService.run:
@Override public void run() { log.info(this.getServiceName() + " service started"); while (!this.isStopped()) { this.waitForRunning(waitInterval); this.mqClientFactory.doRebalance(); } log.info(this.getServiceName() + " service end"); }
至此，producer端的启动流程基本分析完了。总结一下：
- 注意RocketMQTemplate的afterPropertiesSet方法，这里是启动的入口
- 启动主要做了几件事：
- 初始化netty客户端（注意还未链接）
- 启动各种任务，包括nameserver服务地址更新、topic路由规则更新、broker的心跳和离线清理、offset提交任务等。