1. 环境基础
- alluxio环境(及其所依赖的环境)
- hbase环境(及其所依赖的环境)
关于alluxio及hbase等安装部署什么的可以参考本人其他博客
2. 修改配置
官方的介绍是修改hbase的hbase-site.xml文件,但是CDH版不是这么改的,可以在管理面板直接搜索hbase-site
将如下配置增加上
fs.alluxio.impl
alluxio.hadoop.FileSystem
fs.AbstractFileSystem.alluxio.impl
alluxio.hadoop.AlluxioFileSystem
hbase.rootdir
alluxio://:/hbase
3. 分发Alluxio客户端Jar包
将/
开原版的Hbase的lib目录都在hbase的安装目录下,但是CDH版的lib文件夹不太好找,我这里是:
操作完成后重启,官方的说明就只有这么多了
4. 目录说明
hbase之前已经安装好了,使用的hdfs目录. /hbase。 ZooKeeper 中 HBase 的根 znode是/hbase。
alluxio是把全部hdfs挂载进来了,所以alluxio也有hdfs的/hbase目录,本来我打算继续使用这个/hbase目录,希望这样一来hbase中原来的数据可以继续保留在hbase中。
但是重启hbase发现
Failed to become active master
java.io.IOException: alluxio.exception.DirectoryNotEmptyException: Failed to delete /hbase/.tmp/data (UFS dir not in sync. Sync UFS, or delete with unchecked flag.), /hbase/.tmp (Directory not empty) from the under file system
at alluxio.hadoop.AbstractFileSystem.delete(AbstractFileSystem.java:253)
at alluxio.hadoop.FileSystem.delete(FileSystem.java:36)
at org.apache.hadoop.hbase.master.MasterFileSystem.checkTempDir(MasterFileSystem.java:323)
at org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:154)
at org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:122)
at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:864)
at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2260)
at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:583)
at java.lang.Thread.run(Thread.java:748)
Caused by: alluxio.exception.DirectoryNotEmptyException: Failed to delete /hbase/.tmp/data (UFS dir not in sync. Sync UFS, or delete with unchecked flag.), /hbase/.tmp (Directory not empty) from the under file system
at alluxio.client.file.BaseFileSystem.rpc(BaseFileSystem.java:610)
at alluxio.client.file.BaseFileSystem.delete(BaseFileSystem.java:209)
at alluxio.hadoop.AbstractFileSystem.delete(AbstractFileSystem.java:247)
... 8 more
这个目录无法继续使用,于是只能使用新的目录/hbase-alluxio。
此外,如果将这里的目录改为了/hbase-alluxio,那么ooKeeper 中 HBase 的根 znode也要改掉,因为这里面存的是之前的/hbase的信息。、
否则的话,虽然启动能够成功,启动hbase集群输入list,version,status均没有问题,但是当尝试创建表的时候会出现如下错误:
ERROR: org.apache.hadoop.hbase.PleaseHoldException: Master is initializing
at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2998)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1968)
at org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:624)
at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
For usage try 'help "create"'
5. 解决启动失败
按照官方的说明,重启hbase后发现master总是启动失败,查找日志后发现如下:
java.lang.IllegalStateException: The procedure WAL relies on the ability to hsync for proper operation during component failures, but the underlying filesystem does not support doing so. Please check the config value of 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount that can provide it.
at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1080)
at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:423)
at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:600)
at org.apache.hadoop.hbase.master.HMaster.createProcedureExecutor(HMaster.java:1453)
at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:894)
at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2260)
at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:583)
at java.lang.Thread.run(Thread.java:748)
2019-10-12 17:00:04,109 ERROR org.apache.hadoop.hbase.master.HMaster: ***** ABORTING master ahhx-yf-192-168-220-193,16000,1570870798128: Unhandled exception. Starting shutdown. *****
java.lang.IllegalStateException: The procedure WAL relies on the ability to hsync for proper operation during component failures, but the underlying filesystem does not support doing so. Please check the config value of 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount that can provide it.
at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1080)
at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:423)
at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:600)
at org.apache.hadoop.hbase.master.HMaster.createProcedureExecutor(HMaster.java:1453)
at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:894)
at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2260)
at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:583)
at java.lang.Thread.run(Thread.java:748)
解决方案:
继续在刚才配置的地方增加配置
再次重启hbase,发现正常启动了。
6. 测试
启动成功后检查,是有已经转换成了alluxio
alluxio会自动创建出如下目录
数据测试:
准备测试文本
create 'test3', 'cf'
for i in Array(0..99)
put 'test3', 'row'+i.to_s , 'cf:a', 'value'+i.to_s
end
list 'test3'
scan 'test3', {LIMIT => 10, STARTROW => 'row1'}
get 'test3', 'row1'
hbase shell simple_test.txt
sudo -u hdfs hbase org.apache.hadoop.hbase.mapreduce.RowCounter test3
这么修改以后虽然由hdfs转向了alluxio,但是原来hdfs上面的表就看不到了。