hbase异常处理
一、
2014-01-02 02:08:49,349 INFO org.apache.hadoop.hbase.regionserver.Store: Closed info
2014-01-02 02:08:49,349 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed kgroup_user_receiver,66076576,1388481749985.3e0cdf1704a9684c57934c8aca279124.2014-01-02 02:08:49,489 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: stopping server web115,60020,1388476090858; all regions closed.
2014-01-02 02:08:49,490 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: regionserver60020.logSyncer exiting
2014-01-02 02:09:35,823 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Close and delete failed
org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: No lease on /hbase/.logs/web115,60020,1388476090858/web115%2C60020%2C1388476090858.1388596573886: File does not exist. [Lease. Holder: DFSClient_NONMAPREDUCE_1058122613_39, pendingcreates: 1]
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2543)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2535)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:2601)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:2578)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.complete(NameNodeRpcServer.java:556)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.complete(ClientNamenodeProtocolServerSideTranslatorPB.java:337)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:44958)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:453)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1752)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1748)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1746)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:96)
at org.apache.hadoop.hbase.RemoteExceptionHandler.checkThrowable(RemoteExceptionHandler.java:48)
at org.apache.hadoop.hbase.regionserver.HRegionServer.closeWAL(HRegionServer.java:1006)
at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:863)
at java.lang.Thread.run(Thread.java:662)
2014-01-02 02:09:35,824 INFO org.apache.hadoop.hbase.regionserver.Leases: regionserver60020 closing leases
2014-01-02 02:09:35,824 INFO org.apache.hadoop.hbase.regionserver.Leases: regionserver60020 closed leases
2014-01-02 02:09:35,833 WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/rs/web115,60020,1388476090858
2014-01-02 02:09:35,833 INFO org.apache.hadoop.hbase.util.RetryCounter: Sleeping 2000ms before retry #1...
解决办法，调整xcievers参数
默认是4096，改为8192，hdfs-site.xml
dfs.datanode.max.xcievers
8192
二、2014-03-07
2014-03-07 01:24:43,560 INFO org.apache.solr.common.cloud.ConnectionManager: Client is connected to ZooKeeper
2014-03-07 01:24:43,562 INFO org.apache.solr.common.cloud.ZkStateReader: Updating cluster state from ZooKeeper...
2014-03-07 01:24:43,617 INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: Load coprocessor com.baihe.hadoop.hbase.BhdpUserSorlIndexCoprocessor from HTD of kettle_bhdp_user successfully.
2014-03-07 01:24:46,626 INFO org.apache.hadoop.hbase.regionserver.Store: time to purge deletes set to 0ms in store userInfo
2014-03-07 01:24:46,630 INFO org.apache.hadoop.hbase.regionserver.HRegion: Onlined kettle_bhdp_user,,1394126682480.1a8ed89eff93ece7f1c8abb10599479f.; next sequenceid=1
2014-03-07 01:24:46,644 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Post open deploy tasks for region=kettle_bhdp_user,,1394126682480.1a8ed89eff93ece7f1c8abb10599479f., daughter=false
2014-03-07 01:24:46,649 INFO org.apache.hadoop.hbase.catalog.MetaEditor: Updated row kettle_bhdp_user,,1394126682480.1a8ed89eff93ece7f1c8abb10599479f. with server=web116,60020,1394105840490
2014-03-07 01:24:46,649 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Done with post open deploy task for region=kettle_bhdp_user,,1394126682480.1a8ed89eff93ece7f1c8abb10599479f., daughter=false
2014-03-07 01:24:54,125 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Received close region: kettle_bhdp_user,,1394126682480.1a8ed89eff93ece7f1c8abb10599479f.. Version of ZK closing node:0
2014-03-07 01:24:54,126 INFO org.apache.hadoop.hbase.regionserver.Store: Closed userInfo
2014-03-07 01:24:54,780 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server web116,60020,1394105840490: Unrecoverable exception while closing region kettle_bhdp_user,,1394126682480.1a8ed89eff93ece7f1c8abb10599479f., still finishing close
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: missing content stream
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:491)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:197)
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:285)
at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:640)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:68)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:54)
at com.baihe.hadoop.hbase.BhdpUserSorlIndexCoprocessor.stop(BhdpUserSorlIndexCoprocessor.java:125)
at org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.shutdown(CoprocessorHost.java:601)
at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionEnvironment.shutdown(RegionCoprocessorHost.java:118)
at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.shutdown(CoprocessorHost.java:259)
at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postClose(RegionCoprocessorHost.java:341)
at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1050)
at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:957)
at org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:119)
at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:175)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
2014-03-07 01:24:54,781 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: [com.baihe.hadoop.hbase.BhdpUserSorlIndexCoprocessor]
2014-03-07 01:24:54,786 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics: requestsPerSecond=693, numberOfOnlineRegions=103, numberOfStores=233, numberOfStorefiles=414, storefileIndexSizeMB=5, rootIndexSizeKB=5578, totalStaticIndexSizeKB=67905, totalStaticBloomSizeKB=0, memstoreSizeMB=598, mbInMemoryWithoutWAL=8, numberOfPutsWithoutWAL=46070, readRequestsCount=8233420, writeRequestsCount=481536, compactionQueueSize=0, flushQueueSize=0, usedHeapMB=1265, maxHeapMB=2022, blockCacheSizeMB=419.37, blockCacheFreeMB=86.24, blockCacheCount=6472, blockCacheHitCount=10634663, blockCacheMissCount=1366846, blockCacheEvictedCount=1135249, blockCacheHitRatio=88%, blockCacheHitCachingRatio=90%, hdfsBlocksLocalityIndex=48, slowHLogAppendCount=0, fsReadLatencyHistogramMean=4343356.30, fsReadLatencyHistogramCount=148684.00, fsReadLatencyHistogramMedian=163481.00, fsReadLatencyHistogram75th=1317231.50, fsReadLatencyHistogram95th=4640746.35, fsReadLatencyHistogram99th=29729653.39, fsReadLatencyHistogram999th=205902036.17, fsPreadLatencyHistogramMean=19123104.71, fsPreadLatencyHistogramCount=417719.00, fsPreadLatencyHistogramMedian=13272811.00, fsPreadLatencyHistogram75th=29750981.00, fsPreadLatencyHistogram95th=105734844.90, fsPreadLatencyHistogram99th=210200982.64, fsPreadLatencyHistogram999th=728404855.62, fsWriteLatencyHistogramMean=1964821.92, fsWriteLatencyHistogramCount=145665.00, fsWriteLatencyHistogramMedian=273892.50, fsWriteLatencyHistogram75th=351000.50, fsWriteLatencyHistogram95th=2370910.65, fsWriteLatencyHistogram99th=4507722.14, fsWriteLatencyHistogram999th=199344188.82
2014-03-07 01:24:54,789 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Unrecoverable exception while closing region kettle_bhdp_user,,1394126682480.1a8ed89eff93ece7f1c8abb10599479f., still finishing close
2014-03-07 01:24:54,789 ERROR org.apache.hadoop.hbase.executor.EventHandler: Caught throwable while processing event M_RS_CLOSE_REGION
java.lang.RuntimeException: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: missing content stream
at org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:133)
at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:175)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: missing content stream
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:491)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:197)
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:285)
at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:640)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:68)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:54)
at com.baihe.hadoop.hbase.BhdpUserSorlIndexCoprocessor.stop(BhdpUserSorlIndexCoprocessor.java:125)
at org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.shutdown(CoprocessorHost.java:601)
at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionEnvironment.shutdown(RegionCoprocessorHost.java:118)
at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.shutdown(CoprocessorHost.java:259)
at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postClose(RegionCoprocessorHost.java:341)
at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1050)
at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:957)
at org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:119)
... 4 more
2014-03-07 01:24:54,790 INFO org.apache.hadoop.ipc.HBaseServer: Stopping server on 60020
2014-03-07 01:24:54,802 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server listener on 60020
2014-03-07 01:24:54,802 INFO org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 2 on 60020: exiting
2014-03-07 01:24:43,560 INFO org.apache.solr.common.cloud.ConnectionManager: Client is connected to ZooKeeper
2014-03-07 01:24:43,562 INFO org.apache.solr.common.cloud.ZkStateReader: Updating cluster state from ZooKeeper...
2014-03-07 01:24:43,617 INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: Load coprocessor com.baihe.hadoop.hbase.BhdpUserSorlIndexCoprocessor from HTD of kettle_bhdp_user successfully.
2014-03-07 01:24:46,626 INFO org.apache.hadoop.hbase.regionserver.Store: time to purge deletes set to 0ms in store userInfo
2014-03-07 01:24:46,630 INFO org.apache.hadoop.hbase.regionserver.HRegion: Onlined kettle_bhdp_user,,1394126682480.1a8ed89eff93ece7f1c8abb10599479f.; next sequenceid=1
2014-03-07 01:24:46,644 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Post open deploy tasks for region=kettle_bhdp_user,,1394126682480.1a8ed89eff93ece7f1c8abb10599479f., daughter=false
2014-03-07 01:24:46,649 INFO org.apache.hadoop.hbase.catalog.MetaEditor: Updated row kettle_bhdp_user,,1394126682480.1a8ed89eff93ece7f1c8abb10599479f. with server=web116,60020,1394105840490
2014-03-07 01:24:46,649 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Done with post open deploy task for region=kettle_bhdp_user,,1394126682480.1a8ed89eff93ece7f1c8abb10599479f., daughter=false
2014-03-07 01:24:54,125 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Received close region: kettle_bhdp_user,,1394126682480.1a8ed89eff93ece7f1c8abb10599479f.. Version of ZK closing node:0
2014-03-07 01:24:54,126 INFO org.apache.hadoop.hbase.regionserver.Store: Closed userInfo
2014-03-07 01:24:54,780 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server web116,60020,1394105840490: Unrecoverable exception while closing region kettle_bhdp_user,,1394126682480.1a8ed89eff93ece7f1c8abb10599479f., still finishing close
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: missing content stream
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:491)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:197)
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:285)
at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:640)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:68)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:54)
at com.baihe.hadoop.hbase.BhdpUserSorlIndexCoprocessor.stop(BhdpUserSorlIndexCoprocessor.java:125)
at org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.shutdown(CoprocessorHost.java:601)
at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionEnvironment.shutdown(RegionCoprocessorHost.java:118)
at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.shutdown(CoprocessorHost.java:259)
at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postClose(RegionCoprocessorHost.java:341)
at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1050)
at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:957)
at org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:119)
at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:175)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
2014-03-07 01:24:54,781 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: [com.baihe.hadoop.hbase.BhdpUserSorlIndexCoprocessor]
2014-03-07 01:24:54,786 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics: requestsPerSecond=693, numberOfOnlineRegions=103, numberOfStores=233, numberOfStorefiles=414, storefileIndexSizeMB=5, rootIndexSizeKB=5578, totalStaticIndexSizeKB=67905, totalStaticBloomSizeKB=0, memstoreSizeMB=598, mbInMemoryWithoutWAL=8, numberOfPutsWithoutWAL=46070, readRequestsCount=8233420, writeRequestsCount=481536, compactionQueueSize=0, flushQueueSize=0, usedHeapMB=1265, maxHeapMB=2022, blockCacheSizeMB=419.37, blockCacheFreeMB=86.24, blockCacheCount=6472, blockCacheHitCount=10634663, blockCacheMissCount=1366846, blockCacheEvictedCount=1135249, blockCacheHitRatio=88%, blockCacheHitCachingRatio=90%, hdfsBlocksLocalityIndex=48, slowHLogAppendCount=0, fsReadLatencyHistogramMean=4343356.30, fsReadLatencyHistogramCount=148684.00, fsReadLatencyHistogramMedian=163481.00, fsReadLatencyHistogram75th=1317231.50, fsReadLatencyHistogram95th=4640746.35, fsReadLatencyHistogram99th=29729653.39, fsReadLatencyHistogram999th=205902036.17, fsPreadLatencyHistogramMean=19123104.71, fsPreadLatencyHistogramCount=417719.00, fsPreadLatencyHistogramMedian=13272811.00, fsPreadLatencyHistogram75th=29750981.00, fsPreadLatencyHistogram95th=105734844.90, fsPreadLatencyHistogram99th=210200982.64, fsPreadLatencyHistogram999th=728404855.62, fsWriteLatencyHistogramMean=1964821.92, fsWriteLatencyHistogramCount=145665.00, fsWriteLatencyHistogramMedian=273892.50, fsWriteLatencyHistogram75th=351000.50, fsWriteLatencyHistogram95th=2370910.65, fsWriteLatencyHistogram99th=4507722.14, fsWriteLatencyHistogram999th=199344188.82
2014-03-07 01:24:54,789 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Unrecoverable exception while closing region kettle_bhdp_user,,1394126682480.1a8ed89eff93ece7f1c8abb10599479f., still finishing close
2014-03-07 01:24:54,789 ERROR org.apache.hadoop.hbase.executor.EventHandler: Caught throwable while processing event M_RS_CLOSE_REGION
java.lang.RuntimeException: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: missing content stream
at org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:133)
at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:175)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: missing content stream
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:491)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:197)
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:285)
at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:640)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:68)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:54)
at com.baihe.hadoop.hbase.BhdpUserSorlIndexCoprocessor.stop(BhdpUserSorlIndexCoprocessor.java:125)
at org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.shutdown(CoprocessorHost.java:601)
at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionEnvironment.shutdown(RegionCoprocessorHost.java:118)
at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.shutdown(CoprocessorHost.java:259)
at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postClose(RegionCoprocessorHost.java:341)
at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1050)
at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:957)
at org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:119)
... 4 more
2014-03-07 01:24:54,790 INFO org.apache.hadoop.ipc.HBaseServer: Stopping server on 60020
2014-03-07 01:24:54,802 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server listener on 60020
2014-03-07 01:24:54,802 INFO org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 2 on 60020: exiting
三、hbase一台节点高负载异常停止
error or interrupt while splitting logs in
解决方法：
在hbase-site.xml加入一条，让启动hbase集群时不做hlog splitting
参考网址：
http://94it.net/a/jingxuanboke/2014/0721/371385_2.html
http://cdc.zhaopin.com/infomation/Industryinfo/contentinfo?articleid=1713190488&Category=all