hive和hbase的整合

使用hbase作为用户行为分析的数据库，为了提高效率，集成了hive。
[b]安装[/b]
使用cloudera的cdh3u0版本，hbase安装见[url]http://jinghong.iteye.com/blog/1051732[/url]，
解压hive到/home/hadoop/cdh3

tar zxvf hive-0.7.0-cdh3u0.tar.gz -C /home/hadoop/cdh3

在hive/conf目录下复制hive-default.xml到hive-site.xml
修改hive-site.xml，添加

  hive.aux.jars.path
  file:///home/hadoop/cdh3/hive-0.7.0-cdh3u0/lib/hive-hbase-handler-0.7.0-cdh3u0.jar,file:///home/hadoop/cdh3/hive-0.7.0-cdh3u0/lib/hbase-0.90.1-cdh3u0.jar,file:///home/hadoop/cdh3/hive-0.7.0-cdh3u0/lib/zookeeper-3.3.1.jar


  hbase.zookeeper.quorum
  ubuntu-9,ubuntu-10,ubuntu-11
  

注意auxlib的写法，否则按service启动会有如下错误

java.io.IOException: Cannot create an instance of InputSplit class = org.apache.hadoop.hive.hbase.HBaseSplit:org.apache.hadoop.hive.hbase.HBaseSplit

修改hive-site.xml，修改meta源，用mysql作为保存meta的数据库

  javax.jdo.option.ConnectionURL
  jdbc:mysql://192.168.0.53:3306/hive?useUnicode=true&characterEncoding=UTF-8
  JDBC connect string for a JDBC metastore


  javax.jdo.option.ConnectionDriverName
  com.mysql.jdbc.Driver
  Driver class name for a JDBC metastore


  javax.jdo.option.ConnectionUserName
  hive
  username to use against metastore database


  javax.jdo.option.ConnectionPassword
  123456
  password to use against metastore database

注意mysql的hive数据库字符集需要是latin1的。
修改/etc/profile

JAVA_HOME=/usr/local/jdk1.6.0_24
JRE_HOME=$JAVA_HOME/jre
CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATH
HADOOP_HOME=/home/hadoop/cdh3/hadoop-0.20.2-cdh3u0
HBASE_HOME=/home/hadoop/cdh3/hbase-0.90.1-cdh3u0
HIVE_HOME=/home/hadoop/cdh3/hive-0.7.0-cdh3u0
PATH=$JAVA_HOME/bin:$HADOOP_HOME/bin:$HBASE_HOME/bin:$HIVE_HOME/bin:$PATH
export JAVA_HOME JRE_HOME CLASSPATH HADOOP_HOME HBASE_HOME HIVE_HOME PATH  

启动hive

hive

或按service启动

hive --service hiveserver