HBase源码解析(二) HMaster主要类成员解析

本文基于HBase-0.94.1分析HMaster的主要类成员.

HMaster是HBase主/从集群架构中的中央节点。通常一个HBase集群存在多个HMaster节点,其中一个为Active Master,其余为Backup Master.

HMaster的主要类成员如下:

1.ZooKeeper侦听

这些类都继承自ZookeeperListener.

 /******************************ZooKeeperListener and ZooKeeperWatcher*********************************************/
  // Our zk client.
  private ZooKeeperWatcher zooKeeper;
  // Manager and zk listener for master election//继承ZooKeeperListener
  private ActiveMasterManager activeMasterManager;
  // Region server tracker			// 继承ZooKeeperListener
  private RegionServerTracker regionServerTracker;
  // Draining region server tracker//继承ZooKeeperListener
  private DrainingServerTracker drainingServerTracker;
 /** manager of assignment nodes in zookeeper*/
  AssignmentManager assignmentManager;

1.1 ZooKeeperWatcher

ZooKeeperWatcher是HBase里唯一一个实现Watcher接口的类，其他需要感知ZooKeeper的内部类都要通过ZooKeeperWatcher.registerListener(ZooKeeperListener listener)方法向ZooKeeperWatcher实例注册。

zooKeeper是一个ZooKeeperWatcher对象，它在HMaster构造函数中初始化; 它是一个单一的ZooKeeper Watcher ，每个HMaster，RegionServer,Client都会实例化一个ZooKeeperWatcher。 

1.2 ZooKeeperListener

ZooKeeperListener是HBase用来侦听Zookeeper事件的基类。

(1)一个进程的ZookeeperWatcher将执行ZookeeperListener父类的某个方法。为了从watcher收到事件，每个listener必须通过ZookeeperWatcher.registerListener注册自己.

(2)ZooKeeperListener的子类需要重写自己感兴趣的方法 (注意,wather在调用listeners里的方法时将会被阻塞，所以listener里的方法不要long-running)

ActiveMasterManager

该类用来处理master端所有与master选举相关的事情.

(1).侦听并响应关于master znode的zk通知(ZK Notifications),包括nodeCreated和nodeDeleted.

(2).包括一个阻塞方法:持有backup masters，等待active master挂掉.

(3).这个class在HMaster里被初始化,HMaster调用blockUntilBecomingActiveMaster()以阻塞等待成为active master.

RegionServerTracker

该类用来通过ZK跟踪所有OnLine状态的RegionServer

DrainingServerTracker

该类用来通过ZK跟踪所有处于上线/下线流水状态的RegionServer.

AssignmentManager 

assignmentManager 负责管理region的分配工作.

2. RPC Server

HMaster通过HBase RPC机制，将自己封装成一个RPC Server,对外提供RPC调用服务.

  // RPC server for the HMaster
  private final RpcServer rpcServer;

  /**
   * This servers address.
   */
  private final InetSocketAddress isa;

3.HBase文件系统: MasterFileSystem

  MasterFileSystem 抽象了HMaster与低层文件系统交互所需的一系列接口

它在HMaster.finishInitialization()方法中被初始化

  // file system manager for the master FS operations//抽象了HMaster与底层文件系统交互所需的一系列操作
  private MasterFileSystem fileSystemManager

private void finishInitialization(MonitoredTask status, boolean masterRecovery)
  throws IOException, InterruptedException, KeeperException {
	//将是否为active master标识置true
    isActiveMaster = true;
  
    /*我们已经是active master了，开始初始化相关组件.
        ...
    this.masterActiveTime = System.currentTimeMillis();
    // TODO: Do this using Dependency Injection, using PicoContainer, Guice or Spring.
    // 1.创建HBase文件系统 MasterFileSystem.(抽象了HMaster与低层文件系统交互所需的一系列接口)
    this.fileSystemManager = new MasterFileSystem(this, this, metrics, masterRecovery);
        ...
}

4. RegionServer和Catalog管理

ServerManager

ServerManager管理所有的regionServer信息。

实际上，RegionServerTracker,DrainingServerTracker,AssignmentManager 初始化时需要serverManager作为构造函数的参数.

  /******************************ServerManager****************************************************/
  /** server manager to deal with region server info*/
  private ServerManager serverManager;

RegionServerTracker,DrainingServerTracker,AssignmentManager在initializeZKBasedSystemTracker()时进行初始化.

  private void initializeZKBasedSystemTrackers() throws IOException,
      InterruptedException, KeeperException {
    this.catalogTracker = new CatalogTracker(this.zooKeeper, this.conf, this);
    this.catalogTracker.start();

    this.balancer = LoadBalancerFactory.getLoadBalancer(conf);
    this.assignmentManager = new AssignmentManager(this, serverManager,
        this.catalogTracker, this.balancer, this.executorService);
    zooKeeper.registerListenerFirst(assignmentManager);

    this.regionServerTracker = new RegionServerTracker(zooKeeper, this,
        this.serverManager);
    this.regionServerTracker.start();

    this.drainingServerTracker = new DrainingServerTracker(zooKeeper, this,
      this.serverManager);
    this.drainingServerTracker.start();

CatalogTracker 

 catalogTracker 跟踪"目录表"(-ROOT-和.META.表)的可用性

ClusterStatusTracker

clusterStatusTracker跟踪集群在zookeeper上的配置(Tracker on cluster settings up in zookeeper). ClusterStatusTracker和ClusterStatus不同，后者只是一个存储 cluster当前视图的镜像的数据结构，而ClusterStatusTracker是用于跟踪集群在zookeeper上配置的属性信息.

  // manager of catalog regions
  private CatalogTracker catalogTracker;
  // Cluster status zk tracker and local setter
  private ClusterStatusTracker clusterStatusTracker;

5.HBase 执行器服务:ExecutorService

ExecutorService是一个通用的执行器服务类,这个组件抽象了一个threadPool，一个队列，和一个Runnable线程(Handler角色)

  /******************************ExecutorService****************************************************/
  // Instance of the hbase executor service.
  ExecutorService executorService;

该变量在finishInitialization方法中初始化

  private void finishInitialization(MonitoredTask status, boolean masterRecovery)
  throws IOException, InterruptedException, KeeperException {
    ......
    if (!masterRecovery) {
      //初始化ExecutorService和serverManager  
      this.executorService = new ExecutorService(getServerName().toString());//维护一个threadPool和队列
      this.serverManager = new ServerManager(this, this);//管理所有的regionserver
    }

6.负载均衡:LoadBalancer 

 LoadBalancer的职责是维护regionServers之间的负载均衡; balancerChore线程做一些Balancer相关的清理工作.

 用户可以通过实现LoadBalancer接口来定制自己的负载均衡策略. 默认情况下，HBase采用的是org.apache.hadoop.hbase.master.DefaultLoadBalancer类做负载均衡.

  /******************************LoadBalancer, BalancerChore****************************************/
  private LoadBalancer balancer;
  private Thread balancerChore;

loadBalancer是在initializeZKBasedSystemTrackers()方法中调用LoadBalancerFactory.getLoadBalancer(conf)初始化的.

  private void initializeZKBasedSystemTrackers() throws IOException,
      InterruptedException, KeeperException {
    this.balancer = LoadBalancerFactory.getLoadBalancer(conf);
...
}

从LoadBalancerFactory代码可以看出, LoadBalancer由参数HBASE_MASTER_LOADBALANCER_CLASS ( hbase.master.loadbalancer.class)指定，默认值是DefaultLoadBalancer.class

public class LoadBalancerFactory {

  /**
   * Create a loadblanacer from the given conf.
   * @param conf
   * @return A {@link LoadBalancer}
   */
  public static LoadBalancer getLoadBalancer(Configuration conf) {

    // Create the balancer
    Class balancerKlass = conf.getClass(
        HConstants.HBASE_MASTER_LOADBALANCER_CLASS,
        DefaultLoadBalancer.class, LoadBalancer.class);
    return ReflectionUtils.newInstance(balancerKlass, conf);

  }
}

7. Table描述符管理：TableDescriptor

TableDescriptor 接口描述了用来管理Table描述符的一系列操作,FSTableDescriptor是该接口的实现类。

  /******************************TableDescriptors***************************************************/
  private TableDescriptors tableDescriptors;

在HMaster.finishInitialization方法中初始化

 private void finishInitialization(MonitoredTask status, boolean masterRecovery)
  throws IOException, InterruptedException, KeeperException {
    ....
    // 2.初始化tableDescriptors，从文件系统读取HTable的描述信息.
    this.tableDescriptors =
      new FSTableDescriptors(this.fileSystemManager.getFileSystem(),
   this.fileSystemManager.getRootDir());

FTableDescriptor的构造函数如下: FSTableDescriptors根据fs和rootdir可以读取、更改所有的table描述符

  public FSTableDescriptors(final FileSystem fs, final Path rootdir,
      final boolean fsreadOnly) {
    super();
    this.fs = fs;
    this.rootdir = rootdir;
    this.fsreadonly = fsreadOnly;
  }

以FSTableDescriptor。FSTableDescriptor获取rootdir下面的Path(这些Path经过FileUtils.getTableDirs(fs,rootdir)获得，经过了过滤) . 将Path的Name(也即文件名)作为tableName.

  @Override
  public Map getAll()
  throws IOException {
    Map htds = new TreeMap();
    //获取所有的tableName
    List tableDirs = FSUtils.getTableDirs(fs, rootdir);
    for (Path d: tableDirs) { 
    HTableDescriptor htd = null;
   try { 
      htd = get(d.getName());
   } catch (FileNotFoundException fnfe) {
   } if (htd == null)
     continue;
     htds.put(d.getName(), htd);
  } return htds;
 }

8. HTable镜像管理:SnapshotManager

SnapshotManager管理snapshots的生成和装载(taking and restoring)过程.

  // monitor for snapshot of hbase tables
  private SnapshotManager snapshotManager;  //HBase tables的镜像管理者

9.协同处理器管理: CoprocessorProtocol Map

  /******************************protocolHandlers 管理coprocessor protocol的注册*********************/
  // Registered master protocol handlers
  private ClassToInstanceMap
      protocolHandlers = MutableClassToInstanceMap.create();

10.健康检查和垃圾清理: 各种Chore线程

  /** The health check chore. */
  private HealthCheckChore healthCheckChore;//健康检查
  /******************************垃圾回收:CatalogJanitor,LogCleaner,HFileCleaner*********************/
  private CatalogJanitor catalogJanitorChore;  //定时扫描-META-，对无用的region进行垃圾回收
  private LogCleaner logCleaner;
  private HFileCleaner hfileCleaner;

11. 其他

//todo