HRegionServer的初始化到底做了哪些事情,以线程的维度应该能比较清晰的说明,也便于以后更进一步深入的分析。如果能弄清楚每个线程的职责分工,基本上HRegionServer的原理就明白了
HRegionServer的初始化
1.1)实例化
一个典型的栈快照
Thread [main] (Suspended)
WritableRpcEngine$Server(HBaseServer).<init>(String, int, Class<Writable>, int, int, Configuration, String, int) line: 1447
WritableRpcEngine$Server.<init>(Object, Class<?>[], Configuration, String, int, int, int, boolean, int) line: 296
WritableRpcEngine.getServer(Class<VersionedProtocol>, Object, Class<?>[], String, int, int, int, boolean, Configuration, int) line: 245
WritableRpcEngine.getServer(Class, Object, Class[], String, int, int, int, boolean, Configuration, int) line: 55
HBaseRPC.getServer(Class, Object, Class<?>[], String, int, int, int, boolean, Configuration, int) line: 401
HBaseRPC.getServer(Object, Class<?>[], String, int, int, int, boolean, Configuration, int) line: 390
HRegionServer.<init>(Configuration) line: 385
NativeConstructorAccessorImpl.newInstance0(Constructor, Object[]) line: not available [native method]
NativeConstructorAccessorImpl.newInstance(Object[]) line: 39
DelegatingConstructorAccessorImpl.newInstance(Object[]) line: 27
Constructor<T>.newInstance(Object...) line: 513
HRegionServer.constructRegionServer(Class<HRegionServer>, Configuration) line: 3225
HRegionServerCommandLine.start() line: 60
HRegionServerCommandLine.run(String[]) line: 75
ToolRunner.run(Configuration, Tool, String[]) line: 65
HRegionServerCommandLine(ServerCommandLine).doMain(String[]) line: 76
HRegionServer.main(String[]) line: 3251
此时正在实例化HRegionServer
初始化了一个监听线程HBaseServer$Listener
初始化并启动了10个(默认、可调)请求线程HBaseServer$Listener$Reader:
将网络输入字节转化为Call,放入callQueue或者priorityCallQueue
Daemon Thread [IPC Reader 0-9 on port 60020]
初始化一个响应线程HBaseServer$Responder
初始化LruBlockCache
启动缓存逐出线程LruBlockCache$EvictionThread:
平时不干活,当缓存的内存过大会被唤醒,然后开始释放内存但不会使结果小于最小的内存
Daemon Thread [LruBlockCache.EvictionThread]
启动一个定时的缓存使用统计线程LruBlockCache$StatisticsThread
仅仅在debug模式会将缓存统计信息打入日志
Daemon Thread [LRU Statistics #0]
1.2)启动
一个典型的栈快照
Thread [main] (Suspended)
HRegionServer.startRegionServer(HRegionServer, String) line: 3207HRegionServer.main(String[]) line: 3251
启动一个启动线程HRegionServer(本身实现了runnable):
用于向HMaster汇报
Thread [regionserver60020]
初始化zookeeper[HRegionServer.initializeZooKeeper]
启动zookeeper client,实例化ZooKeeper,触发两个线程
ClientCnxn$SendThread
Daemon Thread [regionserver60020-SendThread(vm-cbu-dev-otd-156-53.hst.bjc.kfc.alidc.net:2181)]
ClientCnxn$EventThread
Daemon Thread [regionserver60020-EventThread]
实例化CatalogTracker跟踪-ROOT-和.META.,其中同样会实例化ZooKeeper,触发两个线程
ClientCnxn$SendThread
Daemon Thread [regionserver60020-SendThread(vm-cbu-dev-otd-156-53.hst.bjc.kfc.alidc.net:2181)]
ClientCnxn$EventThread
Daemon Thread [regionserver60020-EventThread]
实例化线程[HRegionServer.initializeThreads]
实例化MemStoreFlusher
实例化CompactSplitThread
实例化Leases线程
Leases线程一种超时检查机制,某些客户调用涉及到耗时操作比如rowlock或者scan,都会在Leases线程中注册,一旦超时,Leases线程会调用一个注册的回调进行处理
在zookeeper注册一个znode
设置复制
实例化Hlog
实例化一个log roll 线程LogRoller:会定期检查log是否超过固定尺寸,超过则roll
实例化Hlog的的SequenceFileLogWriter,
启动一个LeaseCheck线程[org.apache.hadoop.hdfs.DFSClient$LeaseChecker]
(此Lease非彼Lease,前者是haddop,后者属hbase)
Daemon Thread [LeaseChecker]
启动org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer
Daemon Thread [DataStreamer for file /hbase/.logs/vm-cbu-dev-otd-156-53.hst.bjc.kfc.alidc.net,60020,1335341948662/vm-cbu-dev-otd-156-53.hst.bjc.kfc.alidc.net%2C60020%2C1335341948662.1335342838477 block blk_-2454170457116903777_1073]
启动一个Hlog$LogSyncer线程同步writer buffer中的数据
Daemon Thread [regionserver60020.logSyncer]
会触发启动org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$ResponseProcessor线程
Daemon Thread [ResponseProcessor for block blk_-2454170457116903777_1073]
启动服务线程
实例化一堆线程池
启动前面提到的LogRoller线程
Daemon Thread [regionserver60020.logRoller]
启动前面提到的MemStoreFlusher线程
Daemon Thread [regionserver60020.cacheFlusher]
启动前面提到的CompactSplitThread线程
Daemon Thread [regionserver60020.compactionChecker]
启动前面提到的Leases线程
Daemon Thread [regionserver60020.leaseChecker]
启动一个jetty server
启动一个监听线程
Thread [1879358054@qtp-1524522201-0]
启动一个select线程
Thread [1185851599@qtp-1524522201-1 - Acceptor0 [email protected]:60030]
也会触发启动一个java.util.TimerThread,
不知道干嘛个
Daemon Thread [Timer-0]
启动RPC server
启动HBaseServer$Responder线程
Daemon Thread [IPC Server Responder]
启动HBaseServer$Listener线程
Daemon Thread [IPC Server listener on 60020]
启动10个Handler线程:处理callQueue中的请求
Daemon Thread [IPC Server handler 0-9 on 60020]
启动10个优先级Handler线程:处理priorityCallQueue中的请求
Daemon Thread [PRI IPC Server handler 0-9 on 60020]
附上所有线程(见上面标绿的地方)
Java HotSpot(TM) 64-Bit Server VM[10.20.156.53:8000]
Daemon Thread [IPC Reader 0 on port 60020] (Running)