Hive中实现增量更新

现在有一个学生表,其中包括有学生的id,年龄和名称。如下图所示:

hive> select * from student order by id;
OK
1       20      qiu     20140102
2       43      liu     20140102
3       23      zheng   20140102
4       32      yang    20140102
5       24      qian    20140102


根据上流得到的发生改变的数据表为student_temp,字段和属性与student表一致,包含的数据如下所示:

hive> select * from student_temp;
OK
2       10      liu     20140103
3       10      zheng   20140103
Time taken: 0.085 seconds


两个表进行full outer join,将student_temp表中发生修改的数据更新到student表中。

hive> select 
              student.id,
              coalesce(student_temp.age,student.age),
              student.name,coalesce(student_temp.dt,student.dt) 
      from student_temp 
      full outer join student on student_temp.id = student.id;

执行后的结果为:

Ended Job = job_201403041248_0043
MapReduce Jobs Launched: 
Job 0: Map: 2  Reduce: 2   Cumulative CPU: 8.44 sec   HDFS Read: 732 HDFS Write: 356 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 3.68 sec   HDFS Read: 1115 HDFS Write: 94 SUCCESS
Total MapReduce CPU Time Spent: 12 seconds 120 msec
OK
1       20      qiu     20140102
2       10      liu     20140103
3       10      zheng   20140103
4       32      yang    20140102
5       24      qian    20140102
Time taken: 66.223 seconds

作者:liuzhipeng197 发表于2014-3-8 7:46:44 原文链接
阅读:139 评论:0 查看评论

你可能感兴趣的:(hive,更新)