HBase2.4.10 踩坑笔记

安装

目标：安装 HBase2.4.10 ，伪分布式，使用自带的 Zookeeper

参考

1. 确保有 Java 环境

2. 确保有 Hadoop 环境，我的是 Hadoop3.3.1

   [root@main ~]# start-all.sh
   Starting namenodes on [0.0.0.0]
   上一次登录：一 3月 21 18:33:26 CST 2022pts/0 上
   Starting datanodes
   上一次登录：一 3月 21 18:34:12 CST 2022pts/0 上
   Starting secondary namenodes [main]
   上一次登录：一 3月 21 18:34:14 CST 2022pts/0 上
   Starting resourcemanager
   上一次登录：一 3月 21 18:34:23 CST 2022pts/0 上
   Starting nodemanagers
   上一次登录：一 3月 21 18:34:38 CST 2022pts/0 上
   
   [root@main ~]# jps
   81285 NameNode
   85095 Jps
   82118 SecondaryNameNode
   81593 DataNode
   83017 NodeManager
   82749 ResourceManager
   

3. 下载并解压 压缩包(建议HBASE_HOME=/usr/local/hbase)

4. 添加环境变量

   # hbase config
   export HBASE_HOME=/usr/local/hbase
   export PATH=$PATH:$HBASE_HOME/bin
   

5. 修改配置文件hbase/conf/hbase-site.xml,hbase/conf/hbase-env.sh，按自己的需求配

   1. hbase/conf/hbase-site.xml

      <configuration>
        <property>
            <name>hbase.rootdirname>
            <value>hdfs://localhost:9000/hbasevalue>
            <description>hadoop地址和仓库description>
        property>
        <property>
          <name>hbase.cluster.distributedname>
          <value>truevalue>
          <description>是否使用分布式模式（是）description>
        property>
        
        <property>
          <name>hbase.tmp.dirname>
          <value>./tmpvalue>
          <description>临时文件位置description>
        property>
      
        <property>
          <name>hbase.unsafe.stream.capability.enforcename>
          <value>falsevalue>
          <description>是否使用本地文件跑hbase（否，言下之意就是使用hdfs）description>
        property>
      
        <property>
          <name>hbase.zookeeper.quorumname>
          <value>localhost:2181value>
          <description>Zookeeper的节点host:portdescription>
        property>
      configuration>
      

   2. hbase/conf/hbase-env.sh

      
      export JAVA_HOME=/usr/lib/jvm/java
      
      export HBASE_PID_DIR=/var/hbase/pids
      
      # hbase启动时，同时启动自带的zookeeper
      export HBASE_MANAGES_ZK=true
      
      export HBASE_DISABLE_HADOOP_CLASSPATH_LOOKUP="true"
      

6. 常用命令

   # 启动
   start-hbase.sh
   # 关闭
   stop-hbase.sh
   # 查询
   hbase shell
   

   有任何异常记得查看日志文件，在$HBASE_HOME/logs！！！（有master，zookeeper，regionsever）

7. 正常结果

   [root@main ~]# start-hbase.sh
   localhost: running zookeeper, logging to /usr/local/hbase/bin/../logs/hbase-root-zookeeper-main.out
   running master, logging to /usr/local/hbase/logs/hbase-root-master-main.out
   : running regionserver, logging to /usr/local/hbase/logs/hbase-root-regionserver-main.out
   
   [root@main ~]# jps
   14917 Jps
   3478 SecondaryNameNode
   4119 ResourceManager
   4391 NodeManager
   2664 NameNode
   5609 HMaster
   5753 HRegionServer
   5450 HQuorumPeer
   2939 DataNode
   
   
   [root@main ~]# hbase shell
   2022-03-21 19:35:24,229 WARN  [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   HBase Shell
   Use "help" to get list of supported commands.
   Use "exit" to quit this interactive shell.
   For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell
   Version 2.4.10, r3e5359c73d1a96dd7d2ac5bc8f987e9a89ef90ea, Mon Feb 28 10:03:15 PST 2022
   Took 0.0069 seconds                                                                                                                                                                    
   hbase:001:0> create 'user','name','age','addr','phone','email'
   Created table user
   Took 2.7152 seconds                                                                                                                                                                    
   => Hbase::Table - user
   hbase:002:0> list
   TABLE                                                                                                                                                                                  
   user                                                                                                                                                                                   
   1 row(s)
   Took 0.0979 seconds                                                                                                                                                                    
   => ["user"]
   
   [root@main ~]# stop-hbase.sh
   stopping hbase...............
   localhost: running zookeeper, logging to /usr/local/hbase/bin/../logs/hbase-root-zookeeper-main.out
   localhost: stopping zookeeper.
   

Java远程开发

1. 配置IP映射，所有那个远程服务器可能的名称全都配上，包括但不限于 [域名，服务器的原始主机名，自定义的主机名（hadoop，hbase）]

   114.55.58.223 hadoop hbase zookeeper Master iZbp18y7b5jm99960ajdloZ
   第一个是域名，最后一个是主机名，Master是HBase的机器名

2. 将远程服务器上的localhost尽可能替换掉
3. 找到一个教程，林子雨大数据课程文件，点HBase
4. 新建Maven项目，导包，写 ↑ 里的代码并理解，运行查看结果（遇到没有任何输出但不报错，记得打日志）

遇到的问题

1. 执行stop-hbase.sh命令，一直...；执行hbase shell命令，报错Server is not running yet

   [root@main ~]# stop-hbase.sh
   stopping hbase... # 一直.
   
   hbase shell
   # 省略一些启动日志，只留最后一句
   hbase:001:0> create 'user','name','age','addr','phone','email'
   # 报错
   ERROR: org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet
           at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2821)
           at org.apache.hadoop.hbase.master.MasterRpcServices.isMasterRunning(MasterRpcServices.java:1205)
           at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
           at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:392)
           at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
           at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:356)
           at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:336)
   
   For usage try 'help "create"'
   

   查看日志hbase/logs/hbase-root-master-main.out，后搜索

   	2022-03-19 22:57:46,728 WARN  [RS-EventLoopGroup-3-1] concurrent.DefaultPromise: An exception was thrown by org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete()
   java.lang.IllegalArgumentException: object is not an instance of declaring class
   	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   	at java.lang.reflect.Method.invoke(Method.java:498)
   	at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:69)
   

   原因：（我的）Hadoop3.3.1 与 HBase2.4.10 有一点不兼容

   解决办法：hbase/conf/hbase-env.sh取消注释export HBASE_DISABLE_HADOOP_CLASSPATH_LOOKUP="true"

2. hbase shell执行命令，报错ERROR: KeeperErrorCode = ConnectionLoss for /hbase/master

   [root@main ~]# stop-hbase.sh
   no hbase master found
   
   [root@main ~]# hbase shell
   # 省略一些启动日志，只留最后一句
   hbase:001:0> create 'user','name','age','addr','phone','email'
   # 报错
   ERROR: KeeperErrorCode = ConnectionLoss for /hbase/master
   For usage try 'help "create"'
   

   原因：Hadoop处于安全模式
   解决办法：执行hdfs dfsadmin -safemode leave关闭安全模式，可能继续报错(下一个)

3. （这个问题请忽略）start-hbase.sh时 报错，日志包冲突

   [root@main ~]# start-hbase.sh
   SLF4J: Class path contains multiple SLF4J bindings.
   SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
   SLF4J: Found binding in [jar:file:/usr/local/hbase/lib/client-facing-thirdparty/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
   SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
   SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
   running master, logging to /usr/local/hbase/logs/hbase-root-master-main.out
   

   原因：日志包冲突
   解决：把HBase的改名，

   [root@main ~]# cd /usr/local/hbase/lib/client-facing-thirdparty
   [root@main client-facing-thirdparty]# mv slf4j-log4j12-1.7.30.jar slf4j-log4j12-1.7.30.jar.bak
   

4. hadoop与zookeeper连接异常
   表现：进入http://hbase:16010/zk.jsp，最后一行为 stat is not executed because it is not in the whitelist.，（最前面的字符可能为另外的4字符命令）
   原因：命令不在zookeeper白名单中
   解决办法：使用自己的zookeeper，zookeeper配置文件为zoo.cfg（注意最后一行）

   zoo.cfg

   clientPort=2181 
   dataDir=/data 
   dataLogDir=/data/log 
   tickTime=2000 
   initLimit=5 
   syncLimit=2 
   autopurge.snapRetainCount=3 
   autopurge.purgeInterval=0 
   maxClientCnxns=60 
   server.1=whfc.cc:2888:3888
   4lw.commands.whitelist=*
   

5. Java远程操作HBase和Hadoop，admin.tableExists(tableName)语句 不报错也不输出
   原因：IP映射没配置