kylin加载hbas的协处理器(坑)
版本:
CDH: 5.13
Kylin: 2.5.2
在project、model、cube构建完成后，开始查询，结果因为之前做了Hadoop的HA，所以需要更新HBase的协处理器。
这个是执行查询的时候报的错:
org.apache.hadoop.hbase.exceptions.UnknownProtocolException: org.apache.hadoop.hbase.exceptions.UnknownProtocolException: No registered coprocessor service found for name CubeVisitService in region KYLIN_A89D8IUEQJ,,1555293707114.2b06f4a688dd8e3b1c460a02267f395d. at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7972) at org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1986) at org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1968) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33652) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2191) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:183) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:163) while executing SQL: "select * from AAUDIT_TEST02 LIMIT 50000"
在官网找到说明：
http://kylin.apache.org/docs15/howto/howto_update_coprocessor.html
用这个命令来更新HBase的协处理器：
$KYLIN_HOME/bin/kylin.sh org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI $KYLIN_HOME/lib/kylin-coprocessor-*.jar all
但是老报错：
2019-04-15 13:56:50,537 INFO [main-SendThread(data-2.novalocal:2181)] zookeeper.ClientCnxn:975 : Opening socket connection to server data-2.novalocal/172.16.0.18:2181. Will not attempt to authenticate using SASL (unknown error)
2019-04-15 13:56:50,543 INFO [main-SendThread(data-2.novalocal:2181)] zookeeper.ClientCnxn:852 : Socket connection established, initiating session, client: /172.16.0.17:36190, server: data-2.novalocal/172.16.0.18:2181
2019-04-15 13:56:50,551 INFO [main-SendThread(data-2.novalocal:2181)] zookeeper.ClientCnxn:1235 : Session establishment complete on server data-2.novalocal/172.16.0.18:2181, sessionid = 0x369e0f1e2cd8649, negotiated timeout = 60000
2019-04-15 13:56:50,640 INFO [main] util.DeployCoprocessorCLI:107 : Identify coprocessor jar /root/all
2019-04-15 13:56:50,640 INFO [main] util.DeployCoprocessorCLI:114 : Use at most 8 threads to do upgrade
2019-04-15 13:56:50,647 INFO [main] common.KylinConfig:457 : Creating new manager instance of class org.apache.kylin.cube.CubeManager
2019-04-15 13:56:50,658 INFO [main] cube.CubeManager:128 : Initializing CubeManager with config kylin_metadata@hbase
2019-04-15 13:56:50,660 INFO [main] persistence.ResourceStore:88 : Using metadata url kylin_metadata@hbase for resource store
2019-04-15 13:56:50,740 DEBUG [main] hbase.HBaseConnection:180 : Using the working dir FS for HBase: hdfs://sseNameservice
2019-04-15 13:56:50,740 INFO [main] hbase.HBaseConnection:257 : connection is null or closed, creating a new one
2019-04-15 13:56:50,741 INFO [main] zookeeper.RecoverableZooKeeper:120 : Process identifier=hconnection-0x7adbd080 connecting to ZooKeeper ensemble=data-4.novalocal:2181,data-3.novalocal:2181,data-2.novalocal:2181
2019-04-15 13:56:50,741 INFO [main] zookeeper.ZooKeeper:438 : Initiating client connection, connectString=data-4.novalocal:2181,data-3.novalocal:2181,data-2.novalocal:2181 sessionTimeout=60000 watcher=hconnection-0x7adbd0800x0, quorum=data-4.novalocal:2181,data-3.novalocal:2181,data-2.novalocal:2181, baseZNode=/hbase
2019-04-15 13:56:50,742 INFO [main-SendThread(data-3.novalocal:2181)] zookeeper.ClientCnxn:975 : Opening socket connection to server data-3.novalocal/172.16.0.19:2181. Will not attempt to authenticate using SASL (unknown error)
2019-04-15 13:56:50,748 INFO [main-SendThread(data-3.novalocal:2181)] zookeeper.ClientCnxn:852 : Socket connection established, initiating session, client: /172.16.0.17:45920, server: data-3.novalocal/172.16.0.19:2181
2019-04-15 13:56:50,752 INFO [main-SendThread(data-3.novalocal:2181)] zookeeper.ClientCnxn:1235 : Session establishment complete on server data-3.novalocal/172.16.0.19:2181, sessionid = 0x269e0e792d88694, negotiated timeout = 60000
2019-04-15 13:56:51,042 DEBUG [main] hbase.HBaseConnection:306 : HTable 'kylin_metadata' already exists
2019-04-15 13:56:51,111 INFO [main] Configuration.deprecation:1274 : hadoop.native.lib is deprecated. Instead, use io.native.lib.available
2019-04-15 13:56:51,265 DEBUG [main] cachesync.CachedCrudAssist:118 : Reloading CubeInstance from kylin_metadata(key='/cube')@kylin_metadata@hbase
2019-04-15 13:56:51,555 INFO [main] common.KylinConfig:457 : Creating new manager instance of class org.apache.kylin.cube.CubeDescManager
2019-04-15 13:56:51,555 INFO [main] cube.CubeDescManager:87 : Initializing CubeDescManager with config kylin_metadata@hbase
2019-04-15 13:56:51,555 DEBUG [main] cachesync.CachedCrudAssist:118 : Reloading CubeDesc from kylin_metadata(key='/cube_desc')@kylin_metadata@hbase
2019-04-15 13:56:51,627 INFO [main] common.KylinConfig:457 : Creating new manager instance of class org.apache.kylin.metadata.project.ProjectManager
2019-04-15 13:56:51,627 INFO [main] project.ProjectManager:81 : Initializing ProjectManager with metadata url kylin_metadata@hbase
2019-04-15 13:56:51,628 DEBUG [main] cachesync.CachedCrudAssist:118 : Reloading ProjectInstance from kylin_metadata(key='/project')@kylin_metadata@hbase
2019-04-15 13:56:51,657 DEBUG [main] cachesync.CachedCrudAssist:127 : Loaded 3 ProjectInstance(s) out of 3 resource
2019-04-15 13:56:51,658 INFO [main] common.KylinConfig:457 : Creating new manager instance of class org.apache.kylin.metadata.cachesync.Broadcaster
2019-04-15 13:56:51,660 DEBUG [main] cachesync.Broadcaster:101 : 4 nodes in the cluster: [172.16.0.17:7070, 172.16.0.18:7070, 172.16.0.19:7070, 172.16.0.20:7070]
2019-04-15 13:56:51,663 INFO [main] common.KylinConfig:457 : Creating new manager instance of class org.apache.kylin.metadata.model.DataModelManager
2019-04-15 13:56:51,667 INFO [main] common.KylinConfig:457 : Creating new manager instance of class org.apache.kylin.metadata.TableMetadataManager
2019-04-15 13:56:51,668 DEBUG [main] cachesync.CachedCrudAssist:118 : Reloading TableDesc from kylin_metadata(key='/table')@kylin_metadata@hbase
2019-04-15 13:56:51,701 INFO [main] measure.MeasureTypeFactory:117 : Checking custom measure types from kylin config
2019-04-15 13:56:51,702 INFO [main] measure.MeasureTypeFactory:146 : registering COUNT_DISTINCT(hllc), class org.apache.kylin.measure.hllc.HLLCMeasureType$Factory
2019-04-15 13:56:51,706 INFO [main] measure.MeasureTypeFactory:146 : registering COUNT_DISTINCT(bitmap), class org.apache.kylin.measure.bitmap.BitmapMeasureType$Factory
2019-04-15 13:56:51,708 INFO [main] measure.MeasureTypeFactory:146 : registering TOP_N(topn), class org.apache.kylin.measure.topn.TopNMeasureType$Factory
2019-04-15 13:56:51,710 INFO [main] measure.MeasureTypeFactory:146 : registering RAW(raw), class org.apache.kylin.measure.raw.RawMeasureType$Factory
2019-04-15 13:56:51,711 INFO [main] measure.MeasureTypeFactory:146 : registering EXTENDED_COLUMN(extendedcolumn), class org.apache.kylin.measure.extendedcolumn.ExtendedColumnMeasureType$Factory
2019-04-15 13:56:51,713 INFO [main] measure.MeasureTypeFactory:146 : registering PERCENTILE_APPROX(percentile), class org.apache.kylin.measure.percentile.PercentileMeasureType$Factory
2019-04-15 13:56:51,714 INFO [main] measure.MeasureTypeFactory:146 : registering COUNT_DISTINCT(dim_dc), class org.apache.kylin.measure.dim.DimCountDistinctMeasureType$Factory
2019-04-15 13:56:51,721 DEBUG [main] cachesync.CachedCrudAssist:127 : Loaded 4 TableDesc(s) out of 4 resource
2019-04-15 13:56:51,721 DEBUG [main] cachesync.CachedCrudAssist:118 : Reloading TableExtDesc from kylin_metadata(key='/table_exd')@kylin_metadata@hbase
2019-04-15 13:56:51,754 DEBUG [main] cachesync.CachedCrudAssist:127 : Loaded 4 TableExtDesc(s) out of 4 resource
2019-04-15 13:56:51,755 DEBUG [main] cachesync.CachedCrudAssist:118 : Reloading ExternalFilterDesc from kylin_metadata(key='/ext_filter')@kylin_metadata@hbase
2019-04-15 13:56:51,758 DEBUG [main] cachesync.CachedCrudAssist:127 : Loaded 0 ExternalFilterDesc(s) out of 0 resource
2019-04-15 13:56:51,759 DEBUG [main] cachesync.CachedCrudAssist:118 : Reloading DataModelDesc from kylin_metadata(key='/model_desc')@kylin_metadata@hbase
2019-04-15 13:56:51,793 INFO [main] model.DataModelManager:185 : Model demo1_model1 is missing or unloaded yet
2019-04-15 13:56:51,793 INFO [main] model.DataModelManager:185 : Model demo1_model2 is missing or unloaded yet
2019-04-15 13:56:51,799 INFO [main] model.DataModelManager:185 : Model demo1_model2 is missing or unloaded yet
2019-04-15 13:56:51,801 INFO [main] model.DataModelManager:185 : Model hive_test1_demo1 is missing or unloaded yet
2019-04-15 13:56:51,802 INFO [main] model.DataModelManager:185 : Model hive_test2_demo1 is missing or unloaded yet
2019-04-15 13:56:51,802 INFO [main] model.DataModelManager:185 : Model hive_test1_demo3 is missing or unloaded yet
2019-04-15 13:56:51,804 INFO [main] model.DataModelManager:185 : Model hive_test2_demo1 is missing or unloaded yet
2019-04-15 13:56:51,804 DEBUG [main] cachesync.CachedCrudAssist:127 : Loaded 5 DataModelDesc(s) out of 5 resource
2019-04-15 13:56:51,818 DEBUG [main] cachesync.CachedCrudAssist:127 : Loaded 3 CubeDesc(s) out of 3 resource
2019-04-15 13:56:51,821 DEBUG [main] cachesync.CachedCrudAssist:127 : Loaded 3 CubeInstance(s) out of 3 resource
added new table: KYLIN_A89D8IUEQJ
added new table: KYLIN_JQYGCW0X1B
added new table: KYLIN_LHGP2HCHW0
2019-04-15 13:56:51,821 INFO [main] util.DeployCoprocessorCLI:117 : Identify tables [KYLIN_A89D8IUEQJ, KYLIN_JQYGCW0X1B, KYLIN_LHGP2HCHW0]
2019-04-15 13:56:51,821 INFO [main] util.DeployCoprocessorCLI:129 : Tables after filtering by type all: [KYLIN_A89D8IUEQJ, KYLIN_JQYGCW0X1B, KYLIN_LHGP2HCHW0]
2019-04-15 13:56:51,825 INFO [main] util.DeployCoprocessorCLI:181 : Commit Information: 0e519d859e217fbfadd534313376e532d2c647fa;
2019-04-15 13:56:51,831 INFO [main] util.DeployCoprocessorCLI:191 : Filtered tables don't need to deploy coprocessors: [KYLIN_A89D8IUEQJ, KYLIN_JQYGCW0X1B, KYLIN_LHGP2HCHW0]
2019-04-15 13:56:51,831 INFO [main] util.DeployCoprocessorCLI:132 : Will execute tables []
2019-04-15 13:56:51,831 INFO [main] util.DeployCoprocessorCLI:137 : Old coprocessor jar: []
Exception in thread "main" java.io.FileNotFoundException: /root/all (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.(FileInputStream.java:138)
at org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.uploadCoprocessorJar(DeployCoprocessorCLI.java:455)
at org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.main(DeployCoprocessorCLI.java:139)
2019-04-15 13:56:51,901 INFO [close-hbase-conn] hbase.HBaseConnection:136 : Closing HBase connections...
2019-04-15 13:56:51,902 INFO [close-hbase-conn] client.ConnectionManager$HConnectionImplementation:2276 : Closing master protocol: MasterService
2019-04-15 13:56:51,902 INFO [close-hbase-conn] client.ConnectionManager$HConnectionImplementation:1847 : Closing zookeeper sessionid=0x369e0f1e2cd8649
2019-04-15 13:56:51,907 INFO [close-hbase-conn] zookeeper.ZooKeeper:684 : Session: 0x369e0f1e2cd8649 closed
2019-04-15 13:56:51,907 INFO [main-EventThread] zookeeper.ClientCnxn:512 : EventThread shut down
2019-04-15 13:56:51,917 INFO [close-hbase-conn] client.ConnectionManager$HConnectionImplementation:2276 : Closing master protocol: MasterService
2019-04-15 13:56:51,921 INFO [close-hbase-conn] client.ConnectionManager$HConnectionImplementation:1847 : Closing zookeeper sessionid=0x269e0e792d88694
2019-04-15 13:56:51,925 INFO [close-hbase-conn] zookeeper.ZooKeeper:684 : Session: 0x269e0e792d88694 closed
2019-04-15 13:56:51,925 INFO [main-EventThread] zookeeper.ClientCnxn:512 : EventThread shut down
结果就看到2.6.1修复了一个BUG：
新版更新内容有：
[KYLIN-3780] - 在作业信息和邮件通知中新增构建实例
[KYLIN-3794] - TupleFilterVisitor 的 mergeToInClause 某些情况不能正常工作
[KYLIN-3804] - 高级快照（Snapshot）不友好的保存
[KYLIN-3816] - 当前的 CI 不包括流表连接查找表的情况
[KYLIN-3819] - kylin.sh 在前台运行
[KYLIN-3822] - “bin/kylin.sh org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI [file_path] [nOfThread] all” 无法成功运行
[KYLIN-3826] - MergeCuboidJob 仅上传必要的段字典（segment’s dictionary）
此外还有一些 Bug 的修复，更新详情可查阅发布日志
——————————————————————————————————
[KYLIN-3822] - “bin/kylin.sh org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI [file_path] [nOfThread] all” 无法成功运行
换了版本，解决了！！
加一个2.6.1修复的问题：
