Hive安装与配置
Hive的运行需要依赖于Hadoop，因此需要首先安装Hadoop2.5.2，并且Hive的启动前需要首先启动Hadoop。
Hive安装和配置的步骤
1. 从如下地址下载Hive0.14.0
http://mirror.bit.edu.cn/apache/hive/
2.解压hive，在系统变量里配置如下变量HIVE_HOME和PATH
sudo vim /etc/profile
export HIVE_HOME=/home/hadoop/apache-hive-0.14.0-bin
export PATH=$HIVE_HOME/bin:$PATH
3. 修改Hive的配置文件
cp hive-env.sh.template hive-env.sh
4. 修改hive-env.sh中修改如下变量
HADOOP_HOME=/home/hadoop/hadoop-2.5.2
export HIVE_CONF_DIR=/home/hadoop/apache-hive-0.14.0-bin/conf
5.
cp hive.default.template hive-site.xml
编辑hive-site.xml，将其中的变量${system:java.io.tmpdir}${system:user.name}改为如下目录，这个目录需要手工创建，需要特别注意的是system:java.io.tmpdir有多处，需要留意将所有都改掉
/home/hadoop/apache-hive-0.14.0-bin/iotmp
如果不进行这一步，在使用hive命令启动Hive时，会包类似如下错误：
Exception in thread "main" java.lang.RuntimeException: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:444)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:672)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
6. 在Hadoop中执行如下命令用于创建/user/hive/warehouse目录
hdfs dfs -mkdir -p /user/hadoop/warehouse
在HDFS上创建这个目录的原因是在hive-site.xml中，有如下的属性
hive.metastore.warehouse.dir /user/hive/warehouse location of default database for the warehouse
运行Hive
1. 使用hive命令启动Hive
>hive
2. 执行show tables；用于显示当前的表
hive> show tables ;
OK
Time taken: 0.863 seconds
3. 创建简单表：
hive> create table abc(a int,b string);
OK
Time taken: 1.144 second
4. 表操作：
4.1 新建表
hive> create table Word (id int,word string) row format delimited fields terminated by ',' stored as textfile ;
OK
Time taken: 0.153 seconds
新建的表Word，有两个字段id和word，分别是整数和字符串类型，行的格式是使用逗号分隔的文本文件
4.2 在本地系统/user/home/创建一个新文件word.txt, 输入文件
1,MSN
10,QQ
100,Gtalk
1000,Skype
4.3 将word.txt通过Hive上传到HDFS
hive> load data local inpath '/home/hadoop/word.txt' into table Word;
Loading data to table default.word
Table default.word stats: [numFiles=1, totalSize=20]
OK
Time taken: 2.154 seconds
local指明从本地文件系统(/home/hadoop/word.txt)上传数据到Word表中
4.4 查询：select * from Word;
hive> select * from Word;
OK
1 MSN
10 QQ
100 Gtalk
1000 Skype
Time taken: 0.671 seconds, Fetched: 3 row(s)
查询: select * from Word where id = 10;
hive> select * from Word where id = 10;
OK
1 QQ
Time taken: 0.095 seconds, Fetched: 1 row(s)
hive> select * from Word where idNotExist = 1;
FAILED: SemanticException [Error 10004]: Line 1:25 Invalid table alias or column reference 'id1': (possible column names are: id, word)
查看HDFS
1.
hadoop@tom-Inspiron-3521:~/hadoop-2.5.2/bin$ hdfs dfs -ls /user/hive/warehouse
Found 2 items
drwxr-xr-x - hadoop supergroup 0 2014-12-24 23:17 /user/hive/warehouse/abc
drwxr-xr-x - hadoop supergroup 0 2014-12-24 23:28 /user/hive/warehouse/word
abc和word是Hive创建的两个表的表名
2.
hadoop@tom-Inspiron-3521:~/hadoop-2.5.2/bin$ hdfs dfs -ls /user/hive/warehouse/word
Found 1 items
-rw-r--r-- 1 hadoop supergroup 20 2014-12-24 23:28 /user/hive/warehouse/word/word.txt
word表下的word.txt是之前的步骤上传的文件，
3.
hadoop@tom-Inspiron-3521:~/hadoop-2.5.2/bin$ hdfs dfs -cat /user/hive/warehouse/word/word.txt
1 MSN
10 QQ
100 Gtalk
1000 Skype
5.插入数据：
hive> insert into table my_word values(10, "WeChat");
Query ID = hadoop_20150308231111_f2c753b4-e528-4081-887e-cf310dc76695
Total jobs = 3
Launching Job 1 out of 3
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_1425868733189_0001, Tracking URL = http://hadoop.master:8088/proxy/application_1425868733189_0001/
Kill Command = /home/hadoop/software/hadoop-2.5.2/bin/hadoop job -kill job_1425868733189_0001
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2015-03-08 23:11:32,345 Stage-1 map = 0%, reduce = 0%
2015-03-08 23:11:43,706 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 2.23 sec
MapReduce Total cumulative CPU time: 2 seconds 230 msec
Ended Job = job_1425868733189_0001
Stage-4 is selected by condition resolver.
Stage-3 is filtered out by condition resolver.
Stage-5 is filtered out by condition resolver.
Moving data to: hdfs://hadoop.master:9000/tmp/hive/hadoop/70d6d067-6898-4d12-9190-1431ddb4ff9a/hive_2015-03-08_23-11-15_056_4782682719483814130-1/-ext-10000
Loading data to table default.my_word
Table default.my_word stats: [numFiles=2, numRows=1, totalSize=51, rawDataSize=9]
MapReduce Jobs Launched:
Stage-Stage-1: Map: 1 Cumulative CPU: 2.23 sec HDFS Read: 288 HDFS Write: 81 SUCCESS
Total MapReduce CPU Time Spent: 2 seconds 230 msec
OK
Time taken: 30.149 seconds
hive> select * from my_word;
OK
10 WeChat
1 MSN
10 QQ
100 Gtalk
1000 Skype
NULL NULL
Time taken: 0.05 seconds, Fetched: 6 row(s)
插入数据时，HDFS上的变化如下：
[hadoop@hadoop bin]$ ./hdfs dfs -ls /user/hive/warehouse/my_word
Found 2 items
-rw-r--r-- 2 hadoop supergroup 10 2015-03-08 23:11 /user/hive/warehouse/my_word/000000_0
-rw-r--r-- 2 hadoop supergroup 41 2015-03-08 23:09 /user/hive/warehouse/my_word/word.txt
[hadoop@hadoop bin]$ ./hdfs dfs -ls cat /user/hive/warehouse/my_word.txt
ls: `cat': No such file or directory
ls: `/user/hive/warehouse/my_word.txt': No such file or directory
[hadoop@hadoop bin]$ ./hdfs dfs -cat /user/hive/warehouse/my_word.txt
cat: `/user/hive/warehouse/my_word.txt': No such file or directory
[hadoop@hadoop bin]$ ./hdfs dfs -cat /user/hive/warehouse/my_word/word.txt
1,MSN
10,QQ
100,Gtalk
1000,Skype
[hadoop@hadoop bin]$ ./hdfs dfs -cat /user/hive/warehouse/my_word/000000_0
10,WeChat
[hadoop@hadoop bin]$
初始的word.txt没有变化，新增了一个000000_0文件，其中是新增的数据
6. 删除数据
Hive不支持更新操作，delete和update，只能通过其它方式，迂回的完成
hive> delete from my_word where id = 100;
FAILED: SemanticException [Error 10294]: Attempt to do update or delete using transaction manager that does not support these operations.
hive> insert overwrite table my_word select * from my_word where id != 100;
Query ID = hadoop_20150308232020_d8e7491c-006e-4377-8962-8a01dc651a82
Total jobs = 3
Launching Job 1 out of 3
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_1425868733189_0002, Tracking URL = http://hadoop.master:8088/proxy/application_1425868733189_0002/
Kill Command = /home/hadoop/software/hadoop-2.5.2/bin/hadoop job -kill job_1425868733189_0002
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2015-03-08 23:20:34,938 Stage-1 map = 0%, reduce = 0%
2015-03-08 23:20:53,232 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 2.55 sec
MapReduce Total cumulative CPU time: 2 seconds 550 msec
Ended Job = job_1425868733189_0002
Stage-4 is selected by condition resolver.
Stage-3 is filtered out by condition resolver.
Stage-5 is filtered out by condition resolver.
Moving data to: hdfs://hadoop.master:9000/tmp/hive/hadoop/70d6d067-6898-4d12-9190-1431ddb4ff9a/hive_2015-03-08_23-20-20_635_4212544169044890358-1/-ext-10000
Loading data to table default.my_word
Table default.my_word stats: [numFiles=1, numRows=4, totalSize=38, rawDataSize=34]
MapReduce Jobs Launched:
Stage-Stage-1: Map: 1 Cumulative CPU: 2.55 sec HDFS Read: 346 HDFS Write: 109 SUCCESS
Total MapReduce CPU Time Spent: 2 seconds 550 msec
OK
Time taken: 36.279 seconds
hive> select * from my_word;
OK
10 WeChat
1 MSN
10 QQ
1000 Skype
Time taken: 0.086 seconds, Fetched: 4 row(s)
hive>
此时查看HDFS的状态
[hadoop@hadoop bin]$ ./hdfs dfs -ls /user/hive/warehouse/my_word/
Found 1 items
-rw-r--r-- 2 hadoop supergroup 38 2015-03-08 23:20 /user/hive/warehouse/my_word/000000_0
[hadoop@hadoop bin]$ ./hdfs dfs -cat /user/hive/warehouse/my_word/000000_0
10,WeChat
1,MSN
10,QQ
1000,Skype
[hadoop@hadoop bin]$
可见，最初的word.txt已经被覆盖掉，只有000000.0文件，其中不包含id为100的那个记录
a