学习hadoop遇到的问题（hbase整合Phoenix报'hbase.procedure.store.wal.use.hsync'）

从hadoop到hbase的环境搭建一切正常，可直接参照网上教程，这里不累述。

这里记录一下我在配置phoenix连接hbase的时候遇到的问题。
直观的看是HMaster先能启动，过一会又自动挂掉。日志错误信息如下：

ERROR [master/YSL-MASTER:16000] master.HMaster: Failed to become active master
java.lang.IllegalStateException: The procedure WAL relies on the ability to hsync for proper operation during component failures, but the underlying filesystem does not support doing so. Please check the config value of 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount that can provide it.
	at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1043)
	at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:382)
	at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.start(ProcedureExecutor.java:530)
	at org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1222)
	at org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1141)
	at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849)
	at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2019)
	at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:553)
	at java.lang.Thread.run(Thread.java:748)

官网说明：http://hbase.apache.org/book.html#trouble.master.startup.hsync。 看了之后还是不知道怎么解决。

自己尝试了很多方法，换hadoop版本、修改hbase配置，替换hbase/lib目录下hadoop相关jar包(先删除原jar包)，这里记录一下替换命令：find /usr/local/hadoop-2.7.1/share/hadoop -name "hadoop*jar" | xargs -i cp {} /usr/local/hbase-2.0.0/lib/。网上也有说换hbase版本(未尝试)。
网上最多的方法就是：
 

hbase/conf/hbase-site.xml增加配置 

	hbase.unsafe.stream.capability.enforce
	false
	
		Controls whether HBase will check for stream capabilities (hflush/hsync).
	    Disable this if you intend to run on LocalFileSystem, denoted by a rootdir
	    with the 'file://' scheme, but be mindful of the NOTE below.
	    WARNING: Setting this to false blinds you to potential data loss and
	    inconsistent system state in the event of process and/or node failures. If
	    HBase is complaining of an inability to use hsync or hflush it's most
	    likely not a false positive.
    

该配置的作用是：如果你打算在本地文件系统中跑hbase，请禁掉此项。而我这里是将hbase跑在hdfs文件系统上，所以不能配置此项。

我的解决方法：删除每个节点hbase/lib目录下的phoenix-××-client.jar包，重启hbase即可。

 

具体原因我也还没搞清楚，以后再更新。
附上：hadoop-hbase 版本对应关系

记录下hbase/conf/hbase-site.xml中添加的配置：

  	phoenix.schema.isNamespaceMappingEnabled
    true
    开启schema与namespace的对应关系


	phoenix.schema.mapSystemTablesToNamespace
  	true
  	开启schema与namespace的对应关系


    phoenix.functions.allowUserDefinedFunctions
    true
    允许用户自定义函数(UDF)


    hbase.regionserver.wal.codec
    org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec
    开启phoenix 二级索引


    hbase.regionserver.executor.openregion.threads 
    100
    打开用户表region的线程数量(默认3)


    phoenix.default.column.encoded.bytes.attrib
    0
    不对hbase列进行编码存储

转自：https://blog.csdn.net/github_33809414/article/details/86092565