Hbase与Hive整合
Hbase版本0.96.2
Hive版本0.13
在整合的时候需要将Hbase的部分以Hbase开头的包导入到Hive的lib中，并且，尤其注意一个htrace-core-2.04.jar这个jar包，没有的话会出错。
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:3 9)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImp l.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java :387)
... 34 more
Caused by: java.lang.NoClassDefFoundError: org/cloudera/htrace/Trace
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:19 5)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:479)
at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65)
at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:83)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.retrieveCluste rId(HConnectionManager.java:806)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnec tionManager.java:633)
... 39 more
Caused by: java.lang.ClassNotFoundException: org.cloudera.htrace.Trace
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
... 45 more
)
at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:88 )
at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:1 62)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java: 554)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java: 547)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.ja va:89)
at $Proxy11.createTable(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:613)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4194)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:281)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1472)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1239)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1057)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:880)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:870)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)配置文件
hive-site.xml
<property>
<name>hive.aux.jars.path</name>
<value>file:///usr/local/apache-hive-0.13.0-bin/lib/hive-hbase-handler-0.13.0.jar,file:///usr/local/apache-hive-0.13.0-bin/lib/hbase-client-0.96.2-hadoop2.jar,file:///usr/local/apache-hive-0.13.0-bin/lib/hbase-common-0.96.2-hadoop2.jar,file:///usr/local/apache-hive-0.13.0-bin/lib/htrace-core-2.04.jar,file:///usr/local/apache-hive-0.13.0-bin/lib/guava-11.0.2.jar,file:///usr/local/apache-hive-0.13.0-bin/lib/zookeeper-3.4.5.jar</value>
</property>
Hive中建表语句
key部分是行键， col1是列族，value是列。
hive> create table hb_test(key int, value string)
> stored by "org.apache.hadoop.hive.hbase.HBaseStorageHandler"
> with serdeproperties("hbase.columns.mapping"=":key,col1:value")
> tblproperties("hbase.table.name"="hb_test");
Hbase插入数据
hbase(main):066:0> put "hb_test", 2, "col1:value", "wy"
0 row(s) in 0.0090 seconds
hbase(main):067:0> put "hb_test", 3, "col1:value", "w"
0 row(s) in 0.0080 seconds
hbase(main):068:0> put "hb_test", 4, "col1:value", "x"
0 row(s) in 0.0080 seconds
hbase(main):069:0> put "hb_test", 5, "col1:value", "y"
0 row(s) in 0.0070 seconds
hbase(main):070:0> put "hb_test", 11, "col1:value", "ww"
0 row(s) in 0.0080 seconds
hbase(main):071:0> put "hb_test", 22, "col1:value", "yy"
0 row(s) in 0.0100 seconds
hbase(main):072:0> put "hb_test", 33, "col1:value", "x"
0 row(s) in 0.0080 seconds
hbase(main):073:0> put "hb_test", 44, "col1:value", "xx"
0 row(s) in 0.0090 seconds
hbase(main):074:0> put "hb_test", 55, "col1:value", "xx"
0 row(s) in 0.0120 seconds
Hive进行统计查询
hive> select * from hb_test where value = 'x';
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_1482150790771_0001, Tracking URL = http://hadoopwy2:8088/proxy/application_14821507
Kill Command = /usr/local/hadoop2/bin/hadoop job -kill job_1482150790771_0001
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2016-12-19 23:19:07,512 Stage-1 map = 0%, reduce = 0%
2016-12-19 23:20:07,798 Stage-1 map = 0%, reduce = 0%
2016-12-19 23:20:39,077 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 2.96 sec
MapReduce Total cumulative CPU time: 2 seconds 960 msec
Ended Job = job_1482150790771_0001
MapReduce Jobs Launched:
Job 0: Map: 1 Cumulative CPU: 2.96 sec HDFS Read: 261 HDFS Write: 9 SUCCESS
Total MapReduce CPU Time Spent: 2 seconds 960 msec
OK
33 x
4 x
Time taken: 205.946 seconds, Fetched: 2 row(s)
转载注明出处：http://blog.csdn.net/wangyang1354/article/details/53763627