HBase HA高可用集群搭建及HBase Shell简单使用

1.先搭建Hadoop HA集群（包含ZooKeeper搭建），请看这：Hadoop HA高可用集群搭建（2.7.2）

2.我只有五台机器，所以是这么规划的：drguo1/2 HMaster drguo3/4/5 HRegionServer

3.先在drguo1上安装配置好之后，再拷给2/3/4/5。

4.正式开始

4.1去官网下载HBase，最低要0.96版本，之前版本不支持Hadoop HA

4.2加压后配置环境变量

guo@drguo1:~$ sudo gedit /etc/profile

我把我的profile文件贴出来给大家看看

# /etc/profile: system-wide .profile file for the Bourne shell (sh(1))
# and Bourne compatible shells (bash(1), ksh(1), ash(1), ...).

if [ "$PS1" ]; then
  if [ "$BASH" ] && [ "$BASH" != "/bin/sh" ]; then
    # The file bash.bashrc already sets the default PS1.
    # PS1='\h:\w\$ '
    if [ -f /etc/bash.bashrc ]; then
      . /etc/bash.bashrc
    fi
  else
    if [ "`id -u`" -eq 0 ]; then
      PS1='# '
    else
      PS1='$ '
    fi
  fi
fi

# The default umask is now handled by pam_umask.
# See pam_umask(8) and /etc/login.defs.

if [ -d /etc/profile.d ]; then
  for i in /etc/profile.d/*.sh; do
    if [ -r $i ]; then
      . $i
    fi
  done
  unset i
fi
#java
export JAVA_HOME=/opt/Java/jdk1.8.0_73
export JRE_HOME=/opt/Java/jdk1.8.0_73/jre
export CLASSPATH=$JAVA_HOME/lib
export PATH=:$PATH:$JAVA_HOME/bin:$JRE_HOME/bin
#hadoop
export HADOOP_HOME=/opt/Hadoop/hadoop-2.7.2
export PATH=$PATH:$HADOOP_HOME/sbin
export PATH=$PATH:$HADOOP_HOME/bin
#zookeeper
export ZOOKEEPER_HOME=/opt/zookeeper-3.4.8
export PATH=$PATH:$ZOOKEEPER_HOME/bin
#hive
export HIVE_HOME=/opt/apache-hive-2.0.0-bin
export PATH=$PATH:$HIVE_HOME/bin
#hbase
export HBASE_HOME=/opt/hbase

export PATH=$PATH:$HBASE_HOME/bin

4.3修改/opt/hbase/conf/下的hbase-env.sh

guo@drguo1:/opt/hbase/conf$ gedit hbase-env.sh

hbase-env.sh（修改两处）

# The java implementation to use.  Java 1.6 required.
export JAVA_HOME=/opt/Java/jdk1.8.0_73
# Tell HBase whether it should manage it's own instance of Zookeeper or not.不让HBase使用自带的zookeeper，修改为false
export HBASE_MANAGES_ZK=false

4.4修改/opt/hbase/conf/下的hbase-site.xml

guo@drguo1:/opt/hbase/conf$ gedit hbase-site.xml

hbase-site.xml

        hbase.rootdir
        hdfs://ns1/hbase


        hbase.cluster.distributed
        true


        hbase.zookeeper.quorum
        drguo3:2181,drguo4:2181,drguo5:2181

4.5将/opt/Hadoop/hadoop-2.7.2/etc/hadoop下的core-site.xml hdfs-site.xml拷到/opt/hbase/conf/下

4.6将hbase整个文件夹拷给drguo2/3/4/5（拷之前可以把文档删了，这样会快点，好像是hbase下的doc目录，我已经删了，记不清了）

guo@drguo1:~$ scp -r /opt/hbase/ drguo2:/opt/
guo@drguo1:~$ scp -r /opt/hbase/ drguo3:/opt/
guo@drguo1:~$ scp -r /opt/hbase/ drguo4:/opt/
guo@drguo1:~$ scp -r /opt/hbase/ drguo5:/opt/

4.7在启动完zookeeper集群和hadoop集群后启动hbase集群

guo@drguo1:~$ start-hbase.sh 
starting master, logging to /opt/hbase/bin/../logs/hbase-guo-master-drguo1.out
drguo5: starting regionserver, logging to /opt/hbase/bin/../logs/hbase-guo-regionserver-drguo5.out
drguo3: starting regionserver, logging to /opt/hbase/bin/../logs/hbase-guo-regionserver-drguo3.out
drguo4: starting regionserver, logging to /opt/hbase/bin/../logs/hbase-guo-regionserver-drguo4.out
guo@drguo1:~$ jps
3184 DFSZKFailoverController
4211 Jps
4100 HMaster
2796 NameNode
3292 ResourceManager

4.8在drguo2需要单独启动HMaster，当然你也要再配一遍环境变量

guo@drguo2:~$ hbase-daemon.sh start master

4.9查看所有节点的jps，看是否正常启动，也可打开浏览器输入drguo1:60010查看

5.大功告成！

6.HBase Shell简单使用演示

guo@drguo1:~$ hbase shell
2016-03-27 18:07:31,663 INFO  [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
HBase Shell; enter 'help' for list of supported commands.
Type "exit" to leave the HBase Shell
Version 0.96.2-hadoop2, r1581096, Mon Mar 24 16:03:18 PDT 2014

hbase(main):001:0> help
HBase Shell, version 0.96.2-hadoop2, r1581096, Mon Mar 24 16:03:18 PDT 2014
Type 'help "COMMAND"', (e.g. 'help "get"' -- the quotes are necessary) for help on a specific command.
Commands are grouped. Type 'help "COMMAND_GROUP"', (e.g. 'help "general"') for help on a command group.

COMMAND GROUPS:
  Group name: general
  Commands: status, table_help, version, whoami

  Group name: ddl
  Commands: alter, alter_async, alter_status, create, describe, disable, disable_all, drop, drop_all, enable, enable_all, exists, get_table, is_disabled, is_enabled, list, show_filters

  Group name: namespace
  Commands: alter_namespace, create_namespace, describe_namespace, drop_namespace, list_namespace, list_namespace_tables

  Group name: dml
  Commands: count, delete, deleteall, get, get_counter, incr, put, scan, truncate, truncate_preserve

  Group name: tools
  Commands: assign, balance_switch, balancer, catalogjanitor_enabled, catalogjanitor_run, catalogjanitor_switch, close_region, compact, flush, hlog_roll, major_compact, merge_region, move, split, trace, unassign, zk_dump

  Group name: replication
  Commands: add_peer, disable_peer, enable_peer, list_peers, list_replicated_tables, remove_peer

  Group name: snapshot
  Commands: clone_snapshot, delete_snapshot, list_snapshots, rename_snapshot, restore_snapshot, snapshot

  Group name: security
  Commands: grant, revoke, user_permission

SHELL USAGE:
Quote all names in HBase Shell such as table and column names.  Commas delimit
command parameters.  Type  after entering a command to run it.
Dictionaries of configuration used in the creation and alteration of tables are
Ruby Hashes. They look like this:

  {'key1' => 'value1', 'key2' => 'value2', ...}

and are opened and closed with curley-braces.  Key/values are delimited by the
'=>' character combination.  Usually keys are predefined constants such as
NAME, VERSIONS, COMPRESSION, etc.  Constants do not need to be quoted.  Type
'Object.constants' to see a (messy) list of all constants in the environment.

If you are using binary keys or values and need to enter them in the shell, use
double-quote'd hexadecimal representation. For example:

  hbase> get 't1', "key\x03\x3f\xcd"
  hbase> get 't1', "key\003\023\011"
  hbase> put 't1', "test\xef\xff", 'f1:', "\x01\x33\x40"

The HBase shell is the (J)Ruby IRB with the above HBase-specific commands added.
For more on the HBase Shell, see http://hbase.apache.org/docs/current/book.html

建表（必须按它给的范例写！该大写大写！）

hbase(main):007:0> create 'phone', {NAME => 'base_info', VERSIONS => 5}, {NAME => 'extra_info'}
0 row(s) in 8.2020 seconds

=> Hbase::Table - phone
hbase(main):008:0> describe 'phone'
DESCRIPTION                                               ENABLED                       
 'phone', {NAME => 'base_info', BLOOMFILTER => 'ROW', VER true                          
 SIONS => '5', IN_MEMORY => 'false', KEEP_DELETED_CELLS =                               
 > 'false', DATA_BLOCK_ENCODING => 'NONE', TTL => '214748                               
 3647', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCK                               
 CACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE                               
  => '0'}, {NAME => 'extra_info', BLOOMFILTER => 'ROW', V                               
 ERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS                               
  => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL => '2147                               
 483647', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLO                               
 CKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCO                               
 PE => '0'}                                                                             
1 row(s) in 0.3660 seconds

传数据

hbase(main):009:0> put 'phone','001','base_info:brand','oneplus' 
0 row(s) in 0.3230 seconds

hbase(main):010:0> put 'phone','001','base_info:brand','iphone6'
0 row(s) in 0.0250 seconds

查看数据（只显示最新的版本）

hbase(main):011:0> get 'phone','001'
COLUMN                  CELL                                                            
 base_info:brand        timestamp=1459074136116, value=iphone6                          
1 row(s) in 0.0650 seconds

显示base_info列族2个版本数据

hbase(main):012:0> get 'phone','001',{COLUMN => 'base_info:brand',VERSIONS => 2}
COLUMN                  CELL                                                            
 base_info:brand        timestamp=1459074136116, value=iphone6                          
 base_info:brand        timestamp=1459074094569, value=oneplus                          
2 row(s) in 0.0680 seconds

再传几条数据

hbase(main):013:0> put 'phone','001','base_info:brand','iphone7'
0 row(s) in 0.0400 seconds

hbase(main):014:0> put 'phone','001','base_info:brand','iphone8'
0 row(s) in 0.0100 seconds

hbase(main):015:0> put 'phone','001','base_info:brand','iphone9'
0 row(s) in 0.0090 seconds

hbase(main):016:0> put 'phone','001','base_info:brand','iphone10'
0 row(s) in 0.0130 seconds

按理说一共有六条了，但因为之前设置的版本数为5，所以只显示最新的5条

hbase(main):017:0> get 'phone','001',{COLUMN => 'base_info:brand',VERSIONS => 6}
COLUMN                  CELL                                                            
 base_info:brand        timestamp=1459074536842, value=iphone10                         
 base_info:brand        timestamp=1459074518748, value=iphone9                          
 base_info:brand        timestamp=1459074515847, value=iphone8                          
 base_info:brand        timestamp=1459074512440, value=iphone7                          
 base_info:brand        timestamp=1459074136116, value=iphone6                          
5 row(s) in 0.0690 seconds

用scan可以显示6条，因为过时的那条还没有被及时删除

hbase(main):018:0> scan 'phone',{RAW =>true, VERSIONS => 6}
ROW                     COLUMN+CELL                                                     
 001                    column=base_info:brand, timestamp=1459074536842, value=iphone10 
 001                    column=base_info:brand, timestamp=1459074518748, value=iphone9  
 001                    column=base_info:brand, timestamp=1459074515847, value=iphone8  
 001                    column=base_info:brand, timestamp=1459074512440, value=iphone7  
 001                    column=base_info:brand, timestamp=1459074136116, value=iphone6  
 001                    column=base_info:brand, timestamp=1459074094569, value=oneplus  
1 row(s) in 0.1970 seconds

也可用HBase的Java API！如下：

package club.drguo.hbase;

import java.util.ArrayList;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.KeyValue;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.Get;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.util.Bytes;
import org.junit.Before;
import org.junit.Test;

public class HBaseDao {
	private Configuration conf = null;

	@Before
	public void init() {
		conf = HBaseConfiguration.create();
		conf.set("hbase.zookeeper.quorum", "drguo3:2181,drguo4:2181,drguo5:2181");
	}

	// 插入数据
	@Test
	public void putTest() throws Exception {
		HTable phone1 = new HTable(conf, "phone1");
		Put brand = new Put(Bytes.toBytes("001"));
		brand.add(Bytes.toBytes("base_info"), Bytes.toBytes("brand"), Bytes.toBytes("oneplus"));
		Put ram = new Put(Bytes.toBytes("001"));
		ram.add(Bytes.toBytes("base_info"), Bytes.toBytes("ram"), Bytes.toBytes("3GB"));
		ArrayList puts = new ArrayList<>();
		puts.add(brand);
		puts.add(ram);
		phone1.put(puts);
		phone1.close();
	}

	// 查询数据
	@SuppressWarnings("deprecation")
	@Test
	public void testGet() throws Exception {
		HTable table = new HTable(conf, "phone1");
		Get get = new Get(Bytes.toBytes("001"));
		get.setMaxVersions(5);
		Result result = table.get(get);
//		List cells = result.listCells();

		// result.getValue(family, qualifier); 可以从result中直接取出一个特定的value

		// 遍历出result中所有的键值对
		for (KeyValue kv : result.list()) {
			System.out.println(new String(kv.getFamily()));
			System.out.println(new String(kv.getQualifier()));
			System.out.println(new String(kv.getValue()));

		}
		table.close();
	}

	// 建表
	@SuppressWarnings("resource")
	public static void main(String[] args) throws Exception {
		Configuration conf = HBaseConfiguration.create();
		conf.set("hbase.zookeeper.quorum", "drguo3:2181,drguo4:2181,drguo5:2181");
		HBaseAdmin admin = new HBaseAdmin(conf);
		// 指定table名
		TableName tableName = TableName.valueOf("phone1");
		HTableDescriptor tableDescriptordescriptor = new HTableDescriptor(tableName);
		// 指定列族名
		HColumnDescriptor base_info = new HColumnDescriptor("base_info");
		HColumnDescriptor extra_info = new HColumnDescriptor("extra_info");
		// 给base_info设定版本数，不设就是默认数
		base_info.setMaxVersions(5);
		// 加入列族
		tableDescriptordescriptor.addFamily(base_info);
		tableDescriptordescriptor.addFamily(extra_info);
		// 创建表
		admin.createTable(tableDescriptordescriptor);
	}
}