hadoop+zookeeper+hbase伪分布式安装

基本安装步骤

安装包下载

从大数据组件下载地址下载以下组件安装包

1. hadoop-2.6.0-cdh5.6.0.tar.gz
   
2. hbase-1.0.0-cdh5.6.0.tar.gz
   
3. zookeeper-3.4.5-cdh5.6.0.tar.gz
   从java下载地址下载以下组件安装包
   

4. jdk-8u144-linux-x64.tar.gz

   java环境

5. 解压 tar zxvf jdk-8u144-linux-x64.tar.gz
6. 配置环境变量
   1. vim /etc/profile
      

   export JAVA_HOME=/opt/cdh/jdk1.8.0_144
   export JRE_HOME=$JAVA_HOME/jre
   export CLASSPATH=$JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATH
   export PATH=.:$PATH:$JAVA_HOME/bin:$JRE_HOME/bin

   1. source /etc/profile

SSH免密登陆

• 教程链接

hadoop安装

1. 解压 tar zxvf hadoop-2.6.0-cdh5.6.0.tar.gz
2. 修改配置文件,位于目录hadoop-2.6.0-cdh5.6.0/etc/hadoop/下
   • 修改core-site.xml
     

   
    
        hadoop.tmp.dir
        /opt/cdh/hadoop/tmp
    
    
        dfs.name.dir
        /opt/cdh/hadoop/name
    
    
        fs.default.name
        hdfs://ubuntu:9000
    
   

   • 修改mapred-site.xml
     

   
    
        mapred.job.tracker
        ubuntu:9001
    
   

   • 修改hdfs-site.xml

   
     
        dfs.data.dir
        /opt/cdh/hadoop/data
    
    
        dfs.replication
        1
    
    
        dfs.permissions
        false
    
   

   • 修改hadoop-env.sh
     

   export JAVA_HOME=/opt/cdh/jdk1.8.0_144

3. 格式化namenode
   

hadoop-2.6.0-cdh5.6.0/bin/hadoop namenode -format 

1. 启动hdfs
   

hadoop-2.6.0-cdh5.6.0/sbin/start-all.sh

1. 验证
   • jps
   • 登录网址: http://ip:50070

zookeeper安装

1. 解压 tar zxvf zookeeper-3.4.5-cdh5.6.0.tar.gz
2. 修改配置文件,位于目录zookeeper-3.4.5-cdh5.6.0/conf/下
   • zoo.cfg
     

   tickTime=2000
   initLimit=10
   syncLimit=5
   dataDir=/opt/cdh/zookeeper
   clientPort=2181

3. 启动zookeeper
   

zookeeper-3.4.5-cdh5.6.0/bin/zkServer.sh start

hbase安装

1. 解压 tar zxvf hbase-1.0.0-cdh5.6.0.tar.gz
2. 修改配置文件,位于目录hbase-1.0.0-cdh5.6.0/conf/下
   • hbase-env.sh
     

   export JAVA_HOME=/opt/cdh/jdk1.8.0_144
   export HBASE_MANAGES_ZK=false

   • regionservers
     

   ubuntu

   • hbase-site.xml

   
    
        hbase.cluster.distributed
        true    
    
     
        hbase.rootdir 
        hdfs://ubuntu:9000/hbase 
     
     
        hbase.master 
        hdfs://ubuntu:60000 
    
    
        hbase.zookeeper.quorum
        ubuntu:2181
    
    
        hbase.zookeeper.property.dataDir
        /opt/cdh/zookeeper
    
   

3. 启动hbase
   hbase-1.0.0-cdh5.6.0/bin/start-hbase.sh
4. 验证
   1. jps
   2. hbase-1.0.0-cdh5.6.0/bin/hbase shell
   3. 登陆网址： http://ip:60010

IDEA连接hbase登陆

1. 新建maven项目
2. 修改pom.xml文件
   

    
        
            cloudera
            https://repository.cloudera.com/artifactory/cloudera-repos
        
    

    

        
            org.apache.hbase
            hbase
            1.0.0-cdh5.6.0
        

        
            org.apache.hbase
            hbase-client
            1.0.0-cdh5.6.0
        
    

1. 按照百度搜索结果编写demo程序并执行

ERROR解决

1. 先确认以下地址一致
   linux下的/etc/hosts里的ip与windows下C:\Windows\System32\drivers\etc\hosts里的ip映射一致
2. 禁用Hbase自带的zookeeper
   • 修改hbase-env.sh里的export HBASE_MANAGES_ZK=false

转载于:https://www.cnblogs.com/fengzzi/p/10032987.html