HADOOP(2.4)+HBASE(0.98.1)高可用集群详细部署

一、主机分布

节点功能

主机	ZK	JN	zkfc	NN	DN	RM	NM	HM	HR	kdc
Test1	*	*	*	*	*	*	*	*	*
Test2	*	*	*	*	*	*	*	*	*
Test3	*				*		*		*	*

Host

所有节点：

127.0.0.1       localhost

192.168.3.11            test1

192.168.3.12            test2

192.168.3.13            test3

创建ssh信任关系

主节点要信任自己和其它节点，即ssh连接不用密码。

主机test1：

# ssh-keygen    （一路回车）

# ssh-copy-id test1

Are you sure you want to continueconnecting (yes/no)? yes

root@test1's password:

# ssh-copy-id test2

# ssh-copy-id test3

主机test2：

# ssh-keygen    （一路回车）

# ssh-copy-id test2

Are you sure you want to continueconnecting (yes/no)? yes

root@test2's password:

# ssh-copy-id test1

# ssh-copy-id test3

二、Zookeeper

下载

http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz

解压

# tar zxf zookeeper-3.4.6.tar.gz -C/usr/local/

修改配置文件

# cd /usr/local/zookeeper-3.4.6/conf

# cp zoo_sample.cfg zoo.cfg

# vim zoo.cfg

tickTime=2000

initLimit=10

syncLimit=5

dataDir=/var/zookeeper

clientPort=2181

maxClientCnxns=600

server.1=test1:2888:3888

server.2=test2:2888:3888

server.3=test3:2888:3888

zookeeper日志设定

改日志输出位置到/var/log/zookeeper/zookeeper.log,输出和清除方式

# mkdir /var/log/zookeeper

# vim/usr/local/zookeeper-3.4.6/conf/log4j.properties

zookeeper.root.logger=INFO,ROLLINGFILE

zookeeper.log.dir=/var/log/zookeeper

# vim/usr/local/zookeeper-3.4.6/bin/zkEnv.sh

if [ "x${ZOO_LOG_DIR}" ="x" ]

then

   ZOO_LOG_DIR="/var/log/zookeeper"

fi

 

if [ "x${ZOO_LOG4J_PROP}" ="x" ]

then

   ZOO_LOG4J_PROP="INFO,ROLLINGFILE"

Fi

同步配置

主机test1：

# scp -r /usr/local/zookeeper-3.4.6/ test2:/usr/local/

# scp -r /usr/local/zookeeper-3.4.6/ test3:/usr/local/

 

创建myid文件

主机test1：

# echo '1' >>/var/zookeeper/myid

主机test2：

# echo '2' >>/var/zookeeper/myid

主机test3：

# echo '3' >>/var/zookeeper/myid

操作命令

启动：

# /usr/local/zookeeper-3.4.6/bin/zkServer.shstart

加入开机启动：

# echo '/usr/local/zookeeper-3.4.6/bin/zkServer.shstart' >> /etc/rc.local

查看状态：

# /usr/local/zookeeper-3.4.6/bin/zkServer.shstatus

客户端：

# /usr/local/zookeeper-3.4.6/bin/zkCli.sh

三、Hadoop

1、安装

下载

http://lx.cdn.baidupcs.com/file/3278f10a72e52ab2d2d36acc23dab8bb?bkt=p2-nj-321&xcode=a58560088a51fbe91055117b61d7062a3afc73dd9a0ba6cd0b2977702d3e6764&fid=604887638-250528-893812626271274&time=1417490943&sign=FDTAXERB-DCb740ccc5511e5e8fedcff06b081203-IdAFn9T6T2%2BRZpX8CwvI4MdLY58%3D&to=cb&fm=Qin,B,T,t&sta_dx=194&sta_cs=21&sta_ft=gz&sta_ct=6&newver=1&newfm=1&flow_ver=3&expires=8h&rt=sh&r=543216987&mlogid=573369591&vuk=-&vbdid=6575098&fin=hadoop-2.4.0.tar.gz&fn=hadoop-2.4.0.tar.gz

解压

主机test1：

# tar zxf hadoop-2.4.0_x64.tar.gz -C /usr/local/

环境变量

所有节点：

# vim /etc/profile.d/hadoop.sh

export HADOOP_HOME=/usr/local/hadoop-2.4.0/

exportPATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH

exportHADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop/

exportHADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_HOME}/lib/native

exportHADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native"

exportLD_LIBRARY_PATH=$HADOOP_HOME/lib/native

export HADOOP_PID_DIR=/hadoop/pid

export HADOOP_LOG_DIR=/var/log/hadoop

export YARN_PID_DIR=/hadoop/pid

export YARN_LOG_DIR=/var/log/Hadoop

exportHADOOP_MAPRED_LOG_DIR=/var/log/hadoop

export HADOOP_MAPRED_PID_DIR=/hadoop/pid

# source /etc/profile

创建haoop所需目录

所有节点：

# mkdir /Hadoop

# mkdir /var/log/hadoop/

2、配置

配置文件core-site.xml

所有配置文件先配置一台，然后同步到其它主机。

主机test1：

# vim /usr/local/hadoop-2.4.0/etc/hadoop/core-site.xml

<configuration>

<property>

       <name>fs.defaultFS</name>

       <value>hdfs://cluster1</value>

</property>

<property>

       <name>hadoop.tmp.dir</name>

       <value>/hadoop</value>

</property>

<property>

       <name>ha.zookeeper.quorum</name>

       <value>test1:2181,test2:2181,test3:2181</value>       

</property>

</configuration>

配置文件hdfs-site.xml

# vim/usr/local/hadoop-2.4.0/etc/hadoop/hdfs-site.xml

<configuration>

<property>

       <name>dfs.datanode.max.xcievers</name>

       <value>65535</value>

</property>

<property>

       <name>dfs.replication</name>

       <value>1</value>

</property>

<property>

       <name>dfs.nameservices</name>

       <value>cluster1</value>

</property>

<property>

       <name>dfs.ha.namenodes.cluster1</name>

       <value>test1,test2</value>

</property>

<property>

       <name>dfs.namenode.rpc-address.cluster1.test1</name>

       <value>test1:9000</value>

</property>

<property>

       <name>dfs.namenode.http-address.cluster1.test1</name>

       <value>test1:50070</value>

</property>

<property>

       <name>dfs.namenode.rpc-address.cluster1.test2</name>

       <value>test2:9000</value>

</property>

<property>

       <name>dfs.namenode.http-address.cluster1.test2</name>

       <value>test2:50070</value>

</property>

<property>

       <name>dfs.namenode.shared.edits.dir</name>

       <value>qjournal://test1:8485;test2:8485/cluster1</value>

</property>

<property>

       <name>dfs.ha.automatic-failover.enabled</name>

       <value>true</value>

</property>

<property>

       <name>dfs.client.failover.proxy.provider.cluster1</name>

       <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvi

der</value>

</property>

<property>

       <name>dfs.journalnode.edits.dir</name>

       <value>/hadoop/journal</value>

</property>

<property>

       <name>dfs.ha.fencing.methods</name>

       <value>sshfence</value>

</property>

<property>

       <name>dfs.ha.fencing.ssh.private-key-files</name>

       <value>/root/.ssh/id_rsa</value>

</property>

</configuration>

配置文件mapred-site.xml

# vim/usr/local/hadoop-2.4.0/etc/hadoop/mapred-site.xml

<property>

       <name>mapreduce.framework.name</name>

       <value>yarn</value>

</property>

配置文件yarn-site.xml

# vim/usr/local/hadoop-2.4.0/etc/hadoop/yarn-site.xml

<property>

       <name>yarn.nodemanager.aux-services</name>

       <value>mapreduce_shuffle</value>

</property>

<property>

       <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>

       <value>org.apache.hadoop.mapred.ShuffleHandler</value>

</property>

<property>

       <name>yarn.resourcemanager.ha.enabled</name>

       <value>true</value>

</property>

<property>

       <name>yarn.resourcemanager.ha.rm-ids</name>

       <value>rm1,rm2</value>

</property>

<property>

       <name>yarn.resourcemanager.hostname.rm1</name>

       <value>test1</value>

</property>

<property>

       <name>yarn.resourcemanager.hostname.rm2</name>

       <value>test2</value>

</property>

<property>

       <name>yarn.resourcemanager.recovery.enabled</name>

       <value>true</value>

</property>

<property>

       <name>yarn.resourcemanager.store.class</name>

       <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore<

/value>

</property>

<property>

       <name>yarn.resourcemanager.zk-address</name>

       <value>test1:2181,test2:2181,test3:2181</value>

</property>

<property>

       <name>yarn.resourcemanager.cluster-id</name>

       <value>yarn-ha</value>

</property>

配置文件slaves

# vim/usr/local/hadoop-2.4.0/etc/hadoop/slaves

test1

test2

test3

同步配置到其它主机

主机test1：

# scp -r /usr/local/hadoop-2.4.0/ test2:/usr/local/

# scp -r /usr/local/hadoop-2.4.0/ test3:/usr/local/

3、启动

格式化ZooKeeper集群

主机test1：

# hdfs zkfc �CformatZK

# /usr/local/zookeeper-3.4.6/bin/zkCli.sh

 [zk: localhost:2181(CONNECTED) 1] ls /hadoop-ha

[cluster1]

启动JournalNode集群

所有主节点(test1,test2)都启动：

# hadoop-daemon.sh start journalnode

格式化namenode

主机test1：

# hdfs namenode -format -clusterId c1

启动第一个namenode

主机test1：

# hadoop-daemon.sh start namenode

第二个namenode同步元数据

主机test2：

# hdfs namenode �CbootstrapStandby

启动第二个namenode

主机test2：

# hadoop-daemon.sh start namenode

启动ZooKeeperFailoverController

所有主节点都启动：

使namenode一个是active另一个是standby

# hadoop-daemon.sh start zkfc

启动所有datanode

所有从节点(test1,test2,test3)都启动：

# hadoop-daemon.sh start datanode

验证hdfs

# hadoop fs -put xx.txt /

# hdfs dfs -ls /

启动mapreduce

主机test1：

# start-yarn.sh

包括了resourcemanager和所有nodemanager

启动另一个resource-manager

主机test2：

# yarn-daemon.sh start resourcemanager

验证mapreduce

# hadoop jar/usr/local/hadoop-2.4.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.4.0.jar  wordcount  /xx.txt  /out

# hdfs dfs -text /out/part-r-00000

四、Hbase

1、安装

下载hbase并解压

http://mirrors.cnnic.cn/apache/hbase/hbase-0.98.1/hbase-0.98.1-hadoop2-bin.tar.gz

# tar zxf hbase-0.98.1-hadoop2-bin.tar.gz�CC /usr/local/

环境变量

所有节点：

# vim /etc/profile.d/hbase.sh

export HBASE_HOME=/usr/local/hbase-0.98.1

export HBASE_CONF_DIR=$HBASE_HOME/conf

export PATH=$HBASE_HOME/bin:$PATH

export  HBASE_CLASSPATH=$HADOOP_HOME/etc/Hadoop

export  HBASE_LOG_DIR=/var/log/hbase

export  HBASE_PID_DIR=/Hadoop/pid

export HBASE_MANAGES_ZK=false

# source /etc/profile

创建日志目录

所有节点：

# mkdir -p /var/log/hbase

更换haoop2.4的jar包

主机test1：创建一个脚本

# vim lib.sh

#!/bin/sh

hbase_dir=/usr/local/hbase-0.98.1/

hadoop_dir=/usr//local/hadoop-2.4.0/

cd ${hbase_dir}/lib/

find . -name "hadoop*jar" | sed's/2.2.0/2.4.0/g' | sed 's/.\///g' > f.log

rm ./hadoop*jar

cat ./f.log | while read Line

do

find ${hadoop_dir}/share/hadoop -name"$Line" | xargs -i cp {} ./

done

rm ./f.log

# sh lib.sh

# ll /usr/local/hbase-0.98.1/lib/hadoop*jar

2、配置

配置文件hbase-site.xml

所有配置文件先在test1配置，然后同步到其它主机。

主机test1：

<property>

       <name>hbase.rootdir</name>

       <value>hdfs://cluster1/hbase</value>

</property>

<property>

       <name>hbase.cluster.distributed</name>

       <value>true</value>

</property>

<property>

       <name>hbase.zookeeper.quorum</name>

       <value>test1:2181,test2:2181,test3:2181</value>

</property>

配置文件backup-master

# vim/usr/local/hbase-0.98.1/conf/backup-masters

test2

配置文件regionservers

# vim/usr/local/hbase-0.98.1/conf/regionservers

test1

test2

test3

复制配置到其它节点

主机test1：

# scp -r /usr/local/hbase-0.98.1/ test2:/usr/local/

# scp -r /usr/local/hbase-0.98.1/ test3:/usr/local/

3、启动

启动hbase集群

主机test1：

# start-hbase.sh

验证

# jps

2995 HMaster（主节点）

3140 HRegionServer （从节点）