Hive Outline - Part III(Jdbc connect on win/linux)
官方连接: https://cwiki.apache.org/confluence/display/Hive/HiveClient
Hive 环境
CDH 5.0.1
Hadoop 2.3
Hive 0.13.0
Eclipse Helios
OS:Window 7 or CentOS 6.2
判断Hive的metastore类型
Hive的metastore有3种类型，Embeded,Locale,Remote
Embeded就是使用嵌入式进程的本地数据引擎,hive使用的是Derby,我想SqlLite应该也可以。
Locate和Remote我认为本质是一样的，都是使用独立数据引擎来存储metastore.
cdh5.0默认启动了locale 的mysql, 所以可以多个session并发访问。
具体可以去,/hive/lib/conf/hive-site里边查找ConnectionURL和ConnectionDriver
引入依赖的Jar
jdbc的依赖程序，主要来自于hive/lib, hadoop/lib, hadoop/client，所以一般把这几个目录的全部弄出来就可以搞定。
不过具体其实只用到几个，我列举我的成功案例，可以节省看官你的时间。其实这个也是我之前迷惑之处，网上都没有，还是自己动手丰衣足食吧。有人说httpclient不需要，也许，您可以试一下。
07/12/2014 10:11 PM 298,829 commons-configuration-1.6.jar 07/12/2014 10:11 PM 62,050 commons-logging-1.1.3.jar 07/12/2014 10:11 PM 2,827,295 hadoop-common-2.3.0-cdh5.1.0.jar 07/17/2014 08:42 PM 10,023,451 hive-exec-0.12.0-cdh5.1.0.jar 07/17/2014 08:42 PM 132,353 hive-jdbc-0.12.0-cdh5.1.0.jar 07/17/2014 08:42 PM 3,443,238 hive-metastore-0.12.0-cdh5.1.0.jar 07/17/2014 08:42 PM 1,755,877 hive-service-0.12.0-cdh5.1.0.jar 07/12/2014 10:11 PM 433,368 httpclient-4.2.5.jar 07/12/2014 10:11 PM 227,708 httpcore-4.2.5.jar 07/12/2014 11:52 PM 274,725 libfb303-0.9.0.jar 07/12/2014 11:53 PM 348,175 libthrift-0.9.0.cloudera.2.jar 07/12/2014 10:11 PM 489,884 log4j-1.2.17.jar 07/12/2014 10:11 PM 26,084 slf4j-api-1.7.5.jar 07/12/2014 09:42 PM 8,864 slf4j-log4j12-1.7.5.jar
菜花，上程序
iHive有两个server, hiveserver1 和 hiveserver2, 前者未来会废弃。
package com.jinbao.hive.client;
import java.sql.SQLException;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.Statement;
import java.sql.DriverManager;
public class HiveClient2 {
private static String driverName = "org.apache.hive.jdbc.HiveDriver";
public static void main(String[] args) throws SQLException {
System.out.print("Test for access hive by jdbc.\n");
try {
Class.forName(driverName);
} catch (ClassNotFoundException e) {
e.printStackTrace();
System.exit(1);
}
Connection con = DriverManager.getConnection(
"jdbc:hive2://192.168.1.1:10000/default", "hive", "cloudera");
// change to user 'cloudera' if having some problem.
Statement stmt = con.createStatement();
String tableName = "testHiveDriverTable";
stmt.execute("drop table if exists " + tableName);
String create = "create table " + tableName + " (key int, value string)";
create += " row format delimited fields terminated by ','";
stmt.execute(create);
// show tables
String sql = "show tables '" + tableName + "'";
System.out.println("Running: " + sql);
ResultSet res = stmt.executeQuery(sql);
if (res.next()) {
System.out.println(res.getString(1));
}
// describe table
sql = "describe " + tableName;
System.out.println("Running: " + sql);
res = stmt.executeQuery(sql);
while (res.next()) {
System.out.println(res.getString(1) + "\t" + res.getString(2));
}
// load data into table
// NOTE: filepath has to be local to the hive server
// NOTE: /tmp/a.txt is a ctrl-A separated file with two fields per line
String filepath = "/tmp/a.txt";
sql = "load data local inpath '" + filepath + "' into table " + tableName;
System.out.println("Running: " + sql);
stmt.execute(sql);
// select * query
sql = "select * from " + tableName;
System.out.println("Running: " + sql);
res = stmt.executeQuery(sql);
while (res.next()) {
System.out.println(String.valueOf(res.getInt(1)) + "\t" + res.getString(2));
}
// regular hive query
sql = "select count(1) from " + tableName;
System.out.println("Running: " + sql);
res = stmt.executeQuery(sql);
while (res.next()) {
System.out.println(res.getString(1));
}
System.out.print("finally exit.\n");
}
}
SOME NOTE
Hive jdbc 可以执行add jar, create function, show functions, show tables, 但是目前不能执行list jars, source *.hql,等， 所以jdbc的初始化还得一点点地自己加。
最后运行一个复杂的hql查询怎么都报一个错，最后去查yarn resource manager,发现语法分析没过，但是这个语句可以在cli里运行，错误大概是：
return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask.
解决方法，重新启动hive-server2, 命令：server hive-server2 --full-restart
总结:
Hive是个好用,但有时候很慢的好dd.