HBase2.4.10 踩坑笔记

安装

目标:安装 HBase2.4.10 ,伪分布式,使用自带的 Zookeeper

参考

  1. 确保有 Java 环境

  2. 确保有 Hadoop 环境,我的是 Hadoop3.3.1

    [root@main ~]# start-all.sh
    Starting namenodes on [0.0.0.0]
    上一次登录:一 321 18:33:26 CST 2022pts/0 上
    Starting datanodes
    上一次登录:一 321 18:34:12 CST 2022pts/0 上
    Starting secondary namenodes [main]
    上一次登录:一 321 18:34:14 CST 2022pts/0 上
    Starting resourcemanager
    上一次登录:一 321 18:34:23 CST 2022pts/0 上
    Starting nodemanagers
    上一次登录:一 321 18:34:38 CST 2022pts/0 上
    
    [root@main ~]# jps
    81285 NameNode
    85095 Jps
    82118 SecondaryNameNode
    81593 DataNode
    83017 NodeManager
    82749 ResourceManager
    
  3. 下载并解压 压缩包(建议HBASE_HOME=/usr/local/hbase)

  4. 添加环境变量

    # hbase config
    export HBASE_HOME=/usr/local/hbase
    export PATH=$PATH:$HBASE_HOME/bin
    
  5. 修改配置文件hbase/conf/hbase-site.xml,hbase/conf/hbase-env.sh,按自己的需求配

    1. hbase/conf/hbase-site.xml
      <configuration>
        <property>
            <name>hbase.rootdirname>
            <value>hdfs://localhost:9000/hbasevalue>
            <description>hadoop地址和仓库description>
        property>
        <property>
          <name>hbase.cluster.distributedname>
          <value>truevalue>
          <description>是否使用分布式模式(是)description>
        property>
        
        <property>
          <name>hbase.tmp.dirname>
          <value>./tmpvalue>
          <description>临时文件位置description>
        property>
      
        <property>
          <name>hbase.unsafe.stream.capability.enforcename>
          <value>falsevalue>
          <description>是否使用本地文件跑hbase(否,言下之意就是使用hdfs)description>
        property>
      
        <property>
          <name>hbase.zookeeper.quorumname>
          <value>localhost:2181value>
          <description>Zookeeper的节点host:portdescription>
        property>
      configuration>
      
    2. hbase/conf/hbase-env.sh
      
      export JAVA_HOME=/usr/lib/jvm/java
      
      export HBASE_PID_DIR=/var/hbase/pids
      
      # hbase启动时,同时启动自带的zookeeper
      export HBASE_MANAGES_ZK=true
      
      export HBASE_DISABLE_HADOOP_CLASSPATH_LOOKUP="true"
      
  6. 常用命令

    # 启动
    start-hbase.sh
    # 关闭
    stop-hbase.sh
    # 查询
    hbase shell
    

    有任何异常记得查看日志文件,在$HBASE_HOME/logs!!!(有master,zookeeper,regionsever)

  7. 正常结果

    [root@main ~]# start-hbase.sh
    localhost: running zookeeper, logging to /usr/local/hbase/bin/../logs/hbase-root-zookeeper-main.out
    running master, logging to /usr/local/hbase/logs/hbase-root-master-main.out
    : running regionserver, logging to /usr/local/hbase/logs/hbase-root-regionserver-main.out
    
    [root@main ~]# jps
    14917 Jps
    3478 SecondaryNameNode
    4119 ResourceManager
    4391 NodeManager
    2664 NameNode
    5609 HMaster
    5753 HRegionServer
    5450 HQuorumPeer
    2939 DataNode
    
    
    [root@main ~]# hbase shell
    2022-03-21 19:35:24,229 WARN  [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    HBase Shell
    Use "help" to get list of supported commands.
    Use "exit" to quit this interactive shell.
    For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell
    Version 2.4.10, r3e5359c73d1a96dd7d2ac5bc8f987e9a89ef90ea, Mon Feb 28 10:03:15 PST 2022
    Took 0.0069 seconds                                                                                                                                                                    
    hbase:001:0> create 'user','name','age','addr','phone','email'
    Created table user
    Took 2.7152 seconds                                                                                                                                                                    
    => Hbase::Table - user
    hbase:002:0> list
    TABLE                                                                                                                                                                                  
    user                                                                                                                                                                                   
    1 row(s)
    Took 0.0979 seconds                                                                                                                                                                    
    => ["user"]
    
    [root@main ~]# stop-hbase.sh
    stopping hbase...............
    localhost: running zookeeper, logging to /usr/local/hbase/bin/../logs/hbase-root-zookeeper-main.out
    localhost: stopping zookeeper.
    

Java远程开发

  1. 配置IP映射,所有那个远程服务器可能的名称全都配上,包括但不限于 [域名,服务器的原始主机名,自定义的主机名(hadoop,hbase)]

    114.55.58.223 hadoop hbase zookeeper Master iZbp18y7b5jm99960ajdloZ
    第一个是域名,最后一个是主机名,Master是HBase的机器名

  2. 将远程服务器上的localhost尽可能替换掉
  3. 找到一个教程,林子雨大数据课程文件,点HBase
  4. 新建Maven项目,导包,写 ↑ 里的代码并理解,运行查看结果(遇到没有任何输出但不报错,记得打日志)

遇到的问题

  1. 执行stop-hbase.sh命令,一直...;执行hbase shell命令,报错Server is not running yet

    [root@main ~]# stop-hbase.sh
    stopping hbase... # 一直.
    
    hbase shell
    # 省略一些启动日志,只留最后一句
    hbase:001:0> create 'user','name','age','addr','phone','email'
    # 报错
    ERROR: org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet
            at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2821)
            at org.apache.hadoop.hbase.master.MasterRpcServices.isMasterRunning(MasterRpcServices.java:1205)
            at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
            at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:392)
            at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
            at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:356)
            at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:336)
    
    For usage try 'help "create"'
    

    查看日志hbase/logs/hbase-root-master-main.out,后搜索

    	2022-03-19 22:57:46,728 WARN  [RS-EventLoopGroup-3-1] concurrent.DefaultPromise: An exception was thrown by org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete()
    java.lang.IllegalArgumentException: object is not an instance of declaring class
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:498)
    	at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:69)
    

    原因:(我的)Hadoop3.3.1 与 HBase2.4.10 有一点不兼容

    解决办法:hbase/conf/hbase-env.sh取消注释export HBASE_DISABLE_HADOOP_CLASSPATH_LOOKUP="true"

  2. hbase shell执行命令,报错ERROR: KeeperErrorCode = ConnectionLoss for /hbase/master

    [root@main ~]# stop-hbase.sh
    no hbase master found
    
    [root@main ~]# hbase shell
    # 省略一些启动日志,只留最后一句
    hbase:001:0> create 'user','name','age','addr','phone','email'
    # 报错
    ERROR: KeeperErrorCode = ConnectionLoss for /hbase/master
    For usage try 'help "create"'
    

    原因:Hadoop处于安全模式
    解决办法:执行hdfs dfsadmin -safemode leave关闭安全模式,可能继续报错(下一个)

  3. (这个问题请忽略)start-hbase.sh时 报错,日志包冲突

    [root@main ~]# start-hbase.sh
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/usr/local/hbase/lib/client-facing-thirdparty/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
    running master, logging to /usr/local/hbase/logs/hbase-root-master-main.out
    

    原因:日志包冲突
    解决:把HBase的改名,

    [root@main ~]# cd /usr/local/hbase/lib/client-facing-thirdparty
    [root@main client-facing-thirdparty]# mv slf4j-log4j12-1.7.30.jar slf4j-log4j12-1.7.30.jar.bak
    
  4. hadoop与zookeeper连接异常
    表现:进入http://hbase:16010/zk.jsp,最后一行为 stat is not executed because it is not in the whitelist.,(最前面的字符可能为另外的4字符命令)
    原因:命令不在zookeeper白名单中
    解决办法:使用自己的zookeeper,zookeeper配置文件为zoo.cfg(注意最后一行)

    zoo.cfg

    clientPort=2181 
    dataDir=/data 
    dataLogDir=/data/log 
    tickTime=2000 
    initLimit=5 
    syncLimit=2 
    autopurge.snapRetainCount=3 
    autopurge.purgeInterval=0 
    maxClientCnxns=60 
    server.1=whfc.cc:2888:3888
    4lw.commands.whitelist=*
    
  5. Java远程操作HBase和Hadoop,admin.tableExists(tableName)语句 不报错也不输出
    原因:IP映射没配置

你可能感兴趣的:(大数据,java,hbase)