hive 中concat_ws和collect_set 用法

collect_set：对返回的元素集合进行去重返回新的列表，实现列转行。

0: jdbc:hive2://10.67.1.207:10000> select collect_set(cast(ns_hour as string)) as ns_hour from tam_enhance_alarm where ns_date = 20180703;
+-----------------------------------------------------------------------------------------+--+
| ns_hour |
+-----------------------------------------------------------------------------------------+--+
| ["0","1","10","11","12","13","14","15","16","17","18","2","3","4","5","6","7","8","9"] |
+-----------------------------------------------------------------------------------------+--+

concat_ws：字符串连接函数，对生成的列表进行拼接生成新的字符串。
0: jdbc:hive2://10.67.1.207:10000> select concat_ws(',', "a", "b", "c") as ns_hours;
+-----------+--+
| ns_hours |
+-----------+--+
| a,b,c |
+-----------+--+

加入NULL后，自动忽略了：
0: jdbc:hive2://10.67.1.207:10000> select concat_ws(',', "a", NULL, "b", "c") as ns_hours;
+-----------+--+
| ns_hours |
+-----------+--+
| a,b,c |
+-----------+--+

将 collect_set 和 concat_ws 一起用，实现字段元素去重，生成新的记录：
0: jdbc:hive2://10.67.1.207:10000> select concat_ws(',',collect_set(cast(ns_hour as string))) from tam_enhance_alarm where ns_date = 20180703;
+-------------------------------------------------+--+
| _c0 |
+-------------------------------------------------+--+
| 0,1,10,11,12,13,14,15,16,17,18,2,3,4,5,6,7,8,9 |
+-------------------------------------------------+--+

转载于:https://www.cnblogs.com/lihao7/p/9260200.html