系统: LXC - CentOS6.3 x86_64
hadoop版本: cdh5.0.1(manmual安装,未安装cloudera-manager相关)
集群现有环境:node*6;jdk1.7.0_55;已安装zookeeper、hdfs(HA)、yarn、historyserver和httpfs并已开启kerberos(kdc部署在cluster的一个节点上)。
所需安装的包:所有节点-> yum install hbase 主节点-> yum install hbase-master hbase-thrift hbase-rest 从节点-> yum install hbase-regionserver
所需修改的配置文件如下:
主配置文件及内容: /etc/hbase/conf/hbase-site.xml
安全相关配置文件 /etc/hbase/conf/zk-jaas.conf
Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
useTicketCache=false
keyTab="/etc/hadoop/conf/hbase.keytab"
principal="hbase/[email protected]";
};
hbase相关的环境变量文件: /etc/hbase/conf/hbase-env.sh
export HBASE_OPTS="-XX:+UseConcMarkSweepGC"
export HBASE_OPTS="$HBASE_OPTS -Djava.security.auth.login.config=/etc/hbase/conf/zk-jaas.conf"
export HBASE_MANAGES_ZK=false
zookeeper配置文件(仅末尾两行是hbase配置所需追加的): /usr/lib/zookeeper/conf/zoo.cfg
maxClientCnxns=50
tickTime=2000
initLimit=5
syncLimit=2
dataDir=/var/lib/zookeeper
clientPort=2181
server.1=cdh01.barry.com:2888:3888
server.2=cdh02.barry.com:2888:3888
server.3=cdh03.barry.com:2888:3888
authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
jaasLoginRenew=3600000
kerberos.removeHostFromPrincipal=true
kerberos.removeRealmFromPrincipal=true
参考文档说明(hortonworks会简称hdp;cloudera则cdh):
1、系统模板创建,由于笔者在openvz官方搜到的是centos6.5的模板,由于之前在测试环境时尽量和生产保持一致,因此要使用CentOS6.3,所以,笔记根据官方文档自动,自行创建了一个模板,链接如下:http://openvz.org/Creating_a_CentOS_6_Template
2、安装集群参考的cdh官方文档:http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Installation-Guide/CDH5-Installation-Guide.html
3、kerberos和hadoop的整合阶段大量参考了hortonworks的文档,因为cloudera的文档给笔者的感觉就是假设这些都是OK的(简直丧心病狂...),具体链接如下:
hortonworks的kerberos配置链接:http://docs.hortonworks.com/HDPDocuments/Ambari-1.6.1.0/bk_ambari_security/content/index.html
cloudera的kerberos配置链接:http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Security-Guide/CDH5-Security-Guide.html
kerberos官网:http://web.mit.edu/kerberos/krb5-latest/doc/
4、本次主要参考cloudera的文档及hbase官方文档(因为在安装thrift和rest时,cdh未提及相关配置导致无法启动...),相关链接如下:
cdh的文档: http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Installation-Guide/cdh5ig_hbase_installation.html
cdh的hbase和kerberos部署整合的文档:http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Security-Guide/cdh5sg_hbase_security.html
hbase官方文档:http://hbase.apache.org/book/security.html