编写不易,转载请注明(http://shihlei.iteye.com/blog/2081676)!
概述
搭建hbase-0.96.1.1-cdh5.0.1 单机环境,伪分布式,完全分布式
一单机模式安装
在单机模式中,HBase使用本地文件系统,而不是HDFS ,所有的服务和zooKeeper都运作在一个JVM中。zookeep监听一个端口,这样客户端就可以连接HBase了。
(一)程序包
- JRE:jdk1.7.0_25
- HBase:hbase-0.96.1.1-cdh5.0.1
(二)安装
1)解压:tar -xvf hbase-0.96.1.1-cdh5.0.1.tar.gz
source ~/.bashrc
3.修改配置文件:(可选)vi $HBASE_HOME/conf/hbase-site.xml
<configuration>
<property>
<!--hbase data dir,默认: /tmp/hbase-${user.name}-->
<name>hbase.rootdir</name>
<value>file:///home/puppet/hbase/hbase-0.96.1.1-cdh5.0.1/data/hbase</value>
</property>
<property>
<!--zookeeper data write dir,默认:/tmp/hbase-${user.name}-->
<name>hbase.zookeeper.property.dataDir</name>
<value>/home/puppet/hbase/hbase-0.96.1.1-cdh5.0.1/data/zookeeper</value>
</property>
</configuration>
4.启动:start-hbase.sh
5.验证:
(1)进程:jps
-------------------------------------------------------------------------------------------------------------------------------
[puppet@BigData-01 ~]$ jps
24632 HMaster
-------------------------------------------------------------------------------------------------------------------------------
(2)zookeeper端口:netstat -an | grep 2181
(3)shell:hbase shell
-------------------------------------------------------------------------------------------------------------------------------
[puppet@BigData-01 ~]$ hbase shell
2014-06-17 00:09:02,272 INFO [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 0.96.1.1-cdh5.0.1, rUnknown, Tue May 6 13:25:58 PDT 2014
hbase(main):001:0> status
1 servers, 0 dead, 3.0000 average load
-------------------------------------------------------------------------------------------------------------------------------
可以在shell中进行操作:
<a> 表操作:
建表:create '表名' , '列族名'
查看表:discrib '表名'
删除表:drop '表名'
<b>单元操作:
插入:put '表名' , 'row key ' , '列族名:列限定符' , 'value'
删除:delete '表名' , 'row key ' , '列族名:列限定符'
查询:get 坐标
例如:get '表名' , 'rowkey'
(4)页面:
6)停止:stop-hbase.sh
================================================================================
================================================================================
二 伪分布式安装
在为分布式模式中,具有HMaster,HRegionServer进程独立运行,可用于测试,本次搭建独立Zookeeper ,HBase使用 HDFS存储。
(一)规划
1)程序包版本:
- hadoop-2.3.0-cdh5.0.1.tar.gz
- hbase-0.96.1.1-cdh5.0.1.tar.gz
- zookeeper-3.4.5-cdh5.0.1.tar.gz
2) 计划运行进程:
- namenode:hdfs
- datanode:hdfs
- QuorumPeerMain:独立Zookeeper进程
- HMaster:主HMaster进程
- HMaster:备HMaster进程,同一台机器监听其他端口的HMaster进程,一般先启动的是运行的HMaster,是否是运行的收Zookeeper控制。
- HRegionServer
(二)准备:
1)HDFS 伪分布式安装: 可参考《Hadoop-2.3.0-cdh5.0.1单机搭建、伪分布式 ...》
2)Zookeeper单机安装:可参考《Zookeeper-3.4.5-cdh5.0.1 单机模式、副本 ...》
3)解压:tar -xvf hbase-0.96.1.1-cdh5.0.1.tar.gz
4)配置环境变量:vi ~/.bashrc
#HBase
export HBASE_HOME=/home/puppet/hbase/hbase-0.96.1.1-cdh5.0.1
export PATH=$PATH:$HBASE_HOME/bin
source ~/.bashrc
5)修改$HBASE_HOME/conf/hbase-env.sh
export JAVA_HOME="/usr/runtime/java/jdk1.7.0_60"
#//此配置信息,设置由hbase自己管理zookeeper,不需要单独的zookeeper。
export HBASE_MANAGES_ZK=false
6)修改$HBASE_HOME/conf/hbase-site.xml
<configuration>
<property>
<name>hbase.rootdir</name>
<value>hdfs://CentOS-StandAlone:8020/hbase</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>CentOS-StandAlone</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
</configuration>
7)启动:
- 启动HMaster,HRegionServer:start-hbase.sh
- 启动HMaster baskup:local-master-backup.sh start 1
注:
- start-hbase.sh 相当于: hbase-daemon.sh start master ; hbase-daemon.sh start regionserver
- local-master-backup.sh start 1 相当于: 使用hbase-daemon.sh start master 启动且端口使用 默认端口+1k
- 本机启动HRegionServer:local-regionservers.sh start 1 可用于模拟一机多HRegionServer的情况
8)验证:
(1)进程:jps
-------------------------------------------------------------------------------------------------------------------------------
[zero@CentOS-StandAlone bin]$ jps
18470 DataNode
19503 HMaster
18419 NameNode
19656 Jps
15499 QuorumPeerMain
19067 HRegionServer
19306 HMaster
-------------------------------------------------------------------------------------------------------------------------------
注:其他验证同单机模式
附录:
- 停止脚本:stop-hbase.sh
- 停止备份主服务器1:cat /tmp/hbase-${USER}-1-master.pid |xargs kill -9 ;注意 bin/local-master-backup.sh 停止 1 会尝试停止主服务器相关集群。
- 停止单独 regionserver:bin/local-regionservers.sh stop 1
================================================================================
================================================================================
三 完全分布式安装
HBase完全分布式环境搭建,基于CDH5 HDFS NameNode HA ,独立的Zookeeper集群 3台,双HMaster避免单点问题。
(一)规划
1)程序包版本:
- hadoop-2.3.0-cdh5.0.1.tar.gz
- hbase-0.96.1.1-cdh5.0.1.tar.gz
- zookeeper-3.4.5-cdh5.0.1.tar.gz
2)主机规划:
IP |
Host |
部署模块 |
进程 |
8.8.8.11 |
Hadoop-NN-01 |
NameNode HMaster |
NameNode DFSZKFailoverController HMaster |
8.8.8.12 |
Hadoop-NN-02 |
NameNode HMaster |
NameNode DFSZKFailoverController HMaster |
8.8.8.13 |
Hadoop-DN-01 Zookeeper-01 |
DataNode HRegionServer Zookeeper |
DataNode HRegionServer JournalNode QuorumPeerMain |
8.8.8.14 |
Hadoop-DN-02 Zookeeper-02 |
DataNode HRegionServer Zookeeper |
DataNode HRegionServer JournalNode QuorumPeerMain |
8.8.8.15 |
Hadoop-DN-03 Zookeeper-03 |
DataNode HRegionServer Zookeeper |
DataNode HRegionServer JournalNode QuorumPeerMain |
(二)安装:
1)安装Zookeeper 集群:可参考《Zookeeper-3.4.5-cdh5.0.1 单机模式、副本 ...》
2)安装完全分布式HDFS:可参考《Hadoop-2.3.0-cdh5.0.1完全分布式环境搭建 ...》
3)配置互信:配置Zookeeper使用的值Zookeeper-01,Zookeeper-02,Zookeeper-03 使用启动脚本时会要求互信,干脆直接配置好了
ssh-copy-id -i ~/.ssh/id_rsa.pub zero@Zookeeper-01
ssh-copy-id -i ~/.ssh/id_rsa.pub zero@Zookeeper-02
ssh-copy-id -i ~/.ssh/id_rsa.pub zero@Zookeeper-03
4)修改$HBASE_HOME/conf/hbase-env.sh 添加如下:
export JAVA_HOME="/usr/runtime/java/jdk1.7.0_60"
#设置是否使用独立Zookeeper,true为使用独立的Zookeeper
export HBASE_MANAGES_ZK=true
#HBase Options
#export HBASE_OPTS=""
5)修改$HBASE_HOME/conf/hbase-site.xml
这里hbase.rootdir使用Hdfs HA的地址:hdfs://mycluster
<configuration>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.rootdir</name>
<value>hdfs://mycluster/hbase</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>Zookeeper-01:2181,Zookeeper-02:2181,Zookeeper-03:2181</value>
</property>
<!--本地存储目录位于本地文件系统的路径-->
<property>
<name>hbase.local.dir</name>
<value>/home/zero/hbase/hbase-0.96.1.1-cdh5.0.1/data/local</value>
</property>
</configuration>
6)编辑regionservers
-------------------------------------------------------------------------------------------------------------------------------
[zero@CentOS-Cluster-01 conf]$ vi regionservers
Hadoop-DN-01
Hadoop-DN-02
Hadoop-DN-03
-------------------------------------------------------------------------------------------------------------------------------
7)分发程序
-------------------------------------------------------------------------------------------------------------------------------
[zero@CentOS-Cluster-01 conf]$ scp -r hbase-0.96.1.1-cdh5.0.1 zero@Hadoop-NN-02:/home/zero/hbase 100% 39 0.0KB/s 00:00
[zero@CentOS-Cluster-01 conf]$ scp -r hbase-0.96.1.1-cdh5.0.1 zero@Hadoop-DN-01:/home/zero/hbase 100% 39 0.0KB/s 00:00
[zero@CentOS-Cluster-01 conf]$ scp -r hbase-0.96.1.1-cdh5.0.1 zero@Hadoop-DN-02:/home/zero/hbase 100% 39 0.0KB/s 00:00
[zero@CentOS-Cluster-01 conf]$ scp -r hbase-0.96.1.1-cdh5.0.1 zero@Hadoop-DN-03:/home/zero/hbase
-------------------------------------------------------------------------------------------------------------------------------
8)启动
8.8.8.11:启动集群
-------------------------------------------------------------------------------------------------------------------------------
[zero@CentOS-Cluster-01 hbase]$ start-hbase.sh
Zookeeper-03: starting zookeeper, logging to /home/zero/hbase/hbase-0.96.1.1-cdh5.0.1/logs/hbase-zero-zookeeper-CentOS-Cluster-05.out
Zookeeper-02: starting zookeeper, logging to /home/zero/hbase/hbase-0.96.1.1-cdh5.0.1/logs/hbase-zero-zookeeper-CentOS-Cluster-04.out
Zookeeper-01: starting zookeeper, logging to /home/zero/hbase/hbase-0.96.1.1-cdh5.0.1/logs/hbase-zero-zookeeper-CentOS-Cluster-03.out
starting master, logging to /home/zero/hbase/hbase-0.96.1.1-cdh5.0.1/logs/hbase-zero-master-CentOS-Cluster-01.out
Hadoop-DN-03: starting regionserver, logging to /home/zero/hbase/hbase-0.96.1.1-cdh5.0.1/logs/hbase-zero-regionserver-CentOS-Cluster-05.out
Hadoop-DN-01: starting regionserver, logging to /home/zero/hbase/hbase-0.96.1.1-cdh5.0.1/logs/hbase-zero-regionserver-CentOS-Cluster-03.out
Hadoop-DN-02: starting regionserver, logging to /home/zero/hbase/hbase-0.96.1.1-cdh5.0.1/logs/hbase-zero-regionserver-CentOS-Cluster-04.out
-------------------------------------------------------------------------------------------------------------------------------
8.8.8.12:启动备HMaster
-------------------------------------------------------------------------------------------------------------------------------
[zero@CentOS-Cluster-02 conf]$ hbase-daemon.sh start master
starting master, logging to /home/zero/hbase/hbase-0.96.1.1-cdh5.0.1/logs/hbase-zero-master-CentOS-Cluster-02.out
-------------------------------------------------------------------------------------------------------------------------------
9)验证:
进程:
[8.8.8.11,8.8.8.12]:
-------------------------------------------------------------------------------------------------------------------------------
[zero@CentOS-Cluster-01 hbase]$ jps
7454 NameNode
18491 HMaster
18594 Jps
7729 DFSZKFailoverController
-------------------------------------------------------------------------------------------------------------------------------
[8.8.8.13,8.8.8.14,8.8.8.15]:
-------------------------------------------------------------------------------------------------------------------------------
[zero@CentOS-Cluster-03 ~]$ jps
5228 JournalNode
11125 Jps
5137 DataNode
11055 HRegionServer
2514 QuorumPeerMain
-------------------------------------------------------------------------------------------------------------------------------
页面
HBase:
HDFS:
附录:停止
8.8.8.11:stop-hbase.sh
8.8.8.12:hbase-daemon.sh stop master