RocketMQ系列（三）---RocketMQ路由中心NameServer
目录
一、NameServer功能
二、路由中间件设计思路
三、NameServer启动流程
四、路由注册
1、路由元信息
2、路由注册
3、路由删除
4、路由发现
一、NameServer功能
（路由管理，路由注册，服务发现）
1、为了管理和维护配置信息，状态信息，集群中的各个组件，通过它来了解全局信息。
2、各个角色定期向NameServer上报自己的状态，超时未上报的视为不可用
3、NameServer可部署多个，相互独立，每个角色可同时向多个报备状态，达到热备份。nameserver本身无状态，相关组件的状态信息不会持久化，存储在内存中，但它本身支持配置参数的持久化，这个一般用不到。
二、路由中间件设计思路
设计思路：
RocketMQ物理部署图：
RocketMQ物理部署图
NameServer本身可部署多台实现高可用，彼此之间不通信，所以某一时刻二者的数据并不完全相同，但对消息发送和接收无影响，可以通过其他ack机制确保操作成功。
这样设计简单高效，一般NameServer先启动，broker后启动，启动后向所有的NameServer注册，produce发送消息时，会先从NameServer获取可用的broker地址，根据负载算法选择一个并发送。
NameServer与每台Broker服务器保持长连接并间隔30秒，检测Broker是否存活，如果检测到Broker宕机，则从路由注册表中将其移除，但是路由变化不会马上通知消息生产者，这样设计，降低了NameServer实现的复杂性，在消息发送端提供其他容错机制来保证消息发送的高可用性。
三、NameServer启动流程
NameServer源码结构:
NameServer启动源代码：
NamesrvController controller = createNamesrvController(args);
start(controller);public static NamesrvController createNamesrvController(String[] args) throws IOException, JoranException {
//………省略………
//初始化和加载这连个关键的配置
final NamesrvConfig namesrvConfig = new NamesrvConfig();
final NettyServerConfig nettyServerConfig = new NettyServerConfig();
nettyServerConfig.setListenPort(9876);
if (commandLine.hasOption('c')) {
String file = commandLine.getOptionValue('c');
if (file != null) {
InputStream in = new BufferedInputStream(new FileInputStream(file));
properties = new Properties();
properties.load(in);
MixAll.properties2Object(properties, namesrvConfig);
MixAll.properties2Object(properties, nettyServerConfig);
namesrvConfig.setConfigStorePath(file);
System.out.printf("load config properties file OK, %s%n", file);
in.close();
}
}
if (commandLine.hasOption('p')) {
InternalLogger console = InternalLoggerFactory.getLogger(LoggerName.NAMESRV_CONSOLE_NAME);
MixAll.printObjectProperties(console, namesrvConfig);
MixAll.printObjectProperties(console, nettyServerConfig);
System.exit(0);
}
//………省略………
final NamesrvController controller = new NamesrvController(namesrvConfig, nettyServerConfig);
// remember all configs to prevent discard
controller.getConfiguration().registerConfig(properties);
return controller;
} public static NamesrvController start(final NamesrvController controller) throws Exception {
if (null == controller) {
throw new IllegalArgumentException("NamesrvController is null");
}
boolean initResult = controller.initialize();
if (!initResult) {
controller.shutdown();
System.exit(-3);
}
//注册钩子函数，JVM关闭前，关闭线程池
Runtime.getRuntime().addShutdownHook(new ShutdownHookThread(log, new Callable() {
@Override
public Void call() throws Exception {
controller.shutdown();
return null;
}
}));
controller.start();
return controller;
}
1、解析配置文件，填充NameServerConfig、NettyServerConfig属性。
注意：默认监听端口9876.启动NameServer可以用 ./mqnameserver -c configFile -p 打印启动加载的配置属性。
NameServerConfig源码：
private String rocketmqHome = System.getProperty(MixAll.ROCKETMQ_HOME_PROPERTY, System.getenv(MixAll.ROCKETMQ_HOME_ENV));
private String kvConfigPath = System.getProperty("user.home") + File.separator + "namesrv" + File.separator + "kvConfig.json";
private String configStorePath = System.getProperty("user.home") + File.separator + "namesrv" + File.separator + "namesrv.properties";
private String productEnvName = "center";
private boolean clusterTest = false;
//是否开启顺序消息功能，默认为false
private boolean orderMessageEnable = false;NettyServerConfig源码：
//NameServer监听端口，默认初始化9876
private int listenPort = 8888;
//Netty业务线程池里的线程个数
private int serverWorkerThreads = 8;
//public任务线程池线程个数。执行那些未注册线程池的业务
private int serverCallbackExecutorThreads = 0;
//IO线程池线程个数。主要用于处理网络请求，解析，转发等
private int serverSelectorThreads = 3;
//send one way消息请求并发度，broker端使用
private int serverOnewaySemaphoreValue = 256;
//异步消息发送最大并发度。broker端使用。
private int serverAsyncSemaphoreValue = 64;
//网络连接最大空闲时间，超时自动关闭。
private int serverChannelMaxIdleTimeSeconds = 120;
//网络发送端缓存大小
private int serverSocketSndBufSize = NettySystemConfig.socketSndbufSize;
//网络接收端缓存大小
private int serverSocketRcvBufSize = NettySystemConfig.socketRcvbufSize;
//byteBuffer缓存建议开启。
private boolean serverPooledByteBufAllocatorEnable = true;
/**
* make make install
* 是否启用Epoll IO模型。linux下使用。
*
* ../glibc-2.10.1/configure \ --prefix=/usr \ --with-headers=/usr/include \
* --host=x86_64-linux-gnu \ --build=x86_64-pc-linux-gnu \ --without-gd
*/
private boolean useEpollNativeSelector = false;2、初始化NameServer核心控制器NameSrvController
（1）每10s扫描一次Broker，移除不可用的Broker；
（2）每隔10min打印一次KV配置。
初始化boolean initResult = controller.initialize(); 源代码：
public boolean initialize() {
//加载kv配置
this.kvConfigManager.load();
//创建网络对象
this.remotingServer = new NettyRemotingServer(this.nettyServerConfig, this.brokerHousekeepingService);
this.remotingExecutor =
Executors.newFixedThreadPool(nettyServerConfig.getServerWorkerThreads(), new ThreadFactoryImpl("RemotingExecutorThread_"));
this.registerProcessor();
//定时任务一：定时扫描Broker，清除不活动的
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
@Override
public void run() {
NamesrvController.this.routeInfoManager.scanNotActiveBroker();
}
}, 5, 10, TimeUnit.SECONDS);
//定时任务二：定时打印kv配置
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
@Override
public void run() {
NamesrvController.this.kvConfigManager.printAllPeriodically();
}
}, 1, 10, TimeUnit.MINUTES);
// ******省略其他*****
return true;
}3、注册JVM钩子函数，同时启动，监听对应的请求。
四、路由注册
1、路由元信息
RouteInfoManager：
private final static long BROKER_CHANNEL_EXPIRED_TIME = 1000 * 60 * 2;
//可重入读写锁，操作维护元数据信息使用
private final ReadWriteLock lock = new ReentrantReadWriteLock();
//topic信息，Topic名字为key，value的大小等于topic数据存储的master broker个数
//QueuData里包括Broker名称，读队列数量，写队列数量，同步标识
private final HashMap> topicQueueTable;
//broker信息，包含集群名称，broker名称，broker地址（
//Map存储，一主多从，brokerId为key，主都是0，从大于0）
private final HashMap brokerAddrTable;
//集群信息，一个集群下的所有broker
private final HashMap brokerLiveTable;
//过滤服务器。服务端过滤。
private final HashMap/* Filter Server */> filterServerTable;
QueueData：
public class QueueData implements Comparable {
private String brokerName;
private int readQueueNums;
private int writeQueueNums;
private int perm;
private int topicSynFlag;
} BrokerData：
public class BrokerData implements Comparable {
private String cluster;
private String brokerName;
private HashMap BrokerLiveInfo：
class BrokerLiveInfo {
//上次更新时间
private long lastUpdateTimestamp;
private DataVersion dataVersion;
private Channel channel;
private String haServerAddr;
}
2、路由注册
前面说到Broker启动后向所有NameServer发送心跳信息，每隔30S发送自己的心跳包，NameServer收到后更新BrokerLiveTable中的BrokerLiveInfo信息。
public static BrokerController start(BrokerController controller) {
//*****省略*********
controller.start();
//*****省略*********
return null;
}public void start() throws Exception {
//******省略*******
this.registerBrokerAll(true, false, true);
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
@Override
public void run() {
try {
BrokerController.this.registerBrokerAll(true, false, brokerConfig.isForceRegister());
} catch (Throwable e) {
log.error("registerBrokerAll Exception", e);
}
}
}, 1000 * 10, Math.max(10000, Math.min(brokerConfig.getRegisterNameServerPeriod(), 60000)), TimeUnit.MILLISECONDS);
//******省略*******
}NameServer端收到注册请求后执行注册方法,如下图：
这里区分了两个，高版本和低版本不同的方法，两个的差距就在于FilterServer，如下，具体decode里不同实现：
registerBroker里的decode直接就是json转对象；
**withFilterServer多处理了FilterServer相关的内容，读取数据，转化，绑定。
下面重点关注下这个注册过程：
public RegisterBrokerResult registerBroker(
final String clusterName,
final String brokerAddr,
final String brokerName,
final long brokerId,
final String haServerAddr,
final TopicConfigSerializeWrapper topicConfigWrapper,
final List filterServerList,
final Channel channel) {
RegisterBrokerResult result = new RegisterBrokerResult();
try {
try {
this.lock.writeLock().lockInterruptibly();
//获取当前broker
Set brokerNames = this.clusterAddrTable.get(clusterName);
//首次注册需要新建
if (null == brokerNames) {
brokerNames = new HashSet();
this.clusterAddrTable.put(clusterName, brokerNames);
}
brokerNames.add(brokerName);
boolean registerFirst = false;
//获取该Broker所有节点
BrokerData brokerData = this.brokerAddrTable.get(brokerName);
if (null == brokerData) {
registerFirst = true;
brokerData = new BrokerData(clusterName, brokerName, new HashMap());
this.brokerAddrTable.put(brokerName, brokerData);
}
Map brokerAddrsMap = brokerData.getBrokerAddrs();
//Switch slave to master: first remove <1, IP:PORT> in namesrv, then add <0, IP:PORT>
//The same IP:PORT must only have one record in brokerAddrTable
Iterator> it = brokerAddrsMap.entrySet().iterator();
while (it.hasNext()) {
Entry item = it.next();
//这里主要判断如果已经存在该brokerAddr并且key不一致，则移除原来的
if (null != brokerAddr && brokerAddr.equals(item.getValue()) && brokerId != item.getKey()) {
it.remove();
}
}
//更新
String oldAddr = brokerData.getBrokerAddrs().put(brokerId, brokerAddr);
registerFirst = registerFirst || (null == oldAddr);
//如果是主节点
if (null != topicConfigWrapper
&& MixAll.MASTER_ID == brokerId) {
//判断dataVersion 内置时间戳和计数器
if (this.isBrokerTopicConfigChanged(brokerAddr, topicConfigWrapper.getDataVersion())
|| registerFirst) {
ConcurrentMap tcTable =
topicConfigWrapper.getTopicConfigTable();
if (tcTable != null) {
for (Map.Entry entry : tcTable.entrySet()) {
//创建或更新Topic
this.createAndUpdateQueueData(brokerName, entry.getValue());
}
}
}
}
//更新BrokerLiveInfo
BrokerLiveInfo prevBrokerLiveInfo = this.brokerLiveTable.put(brokerAddr,
new BrokerLiveInfo(
System.currentTimeMillis(),
topicConfigWrapper.getDataVersion(),
channel,
haServerAddr));
if (null == prevBrokerLiveInfo) {
log.info("new broker registered, {} HAServer: {}", brokerAddr, haServerAddr);
}
//添加filterServer
if (filterServerList != null) {
if (filterServerList.isEmpty()) {
this.filterServerTable.remove(brokerAddr);
} else {
this.filterServerTable.put(brokerAddr, filterServerList);
}
}
//绑定主节点和集群地址
if (MixAll.MASTER_ID != brokerId) {
String masterAddr = brokerData.getBrokerAddrs().get(MixAll.MASTER_ID);
if (masterAddr != null) {
BrokerLiveInfo brokerLiveInfo = this.brokerLiveTable.get(masterAddr);
if (brokerLiveInfo != null) {
result.setHaServerAddr(brokerLiveInfo.getHaServerAddr());
result.setMasterAddr(masterAddr);
}
}
}
} finally {
this.lock.writeLock().unlock();
}
} catch (Exception e) {
log.error("registerBroker Exception", e);
}
return result;
} 至此，registerBroker过程就完毕了。
3、路由删除
4、路由发现