继承udf这个类,方法重载evaluate
1. add jar /opt/udftest.jar
2. create temporary function 功能名as “主类路径”;
1. add FILE weekday_mapper.py;
2. SELECT TRANSFORM (必须是所有的字段,或者*)USING ‘python 脚本’ AS (生成的字段名)FROM t_rating;
数据:
{"movie":"3664","rate":"4","timeStamp":"961685303","uid":"5225"}
新建一张表加载进去数据
create table temp(line string);
脚本:
#!/bin/python
import sys
import datetime
import json
for line in sys.stdin:
if line!='\n':
result=json.loads(line.strip())
movie=result['movie']
rate=result['rate']
uid=result['uid']
weekday =datetime.datetime.fromtimestamp(float(result['timeStamp'])).isoweekday()
print('\t'.join([movie,rate,str(weekday),uid]))
使用脚本:
add FILE weekday.py;
select transform (line) using 'python weekday.py' as (movie,rate,wek,id) from temp;
可以参考:https://blog.csdn.net/u013980127/article/details/52606024
1. row_number() over(partition by fields order by fields):
用于对返回的字段结果进行分组排序,并编号
例如:
```
select id,name,dt,rank from (select id,name,dt,row_number() over(partition by country order by dt) as rank from person) as tmp where tmp.rank<3;
```
结果:
![在这里插入图片描述](https://img-blog.csdnimg.cn/20190514153419541.png)
2. 日期函数:
• 日期比较函数 : datediff
语法: datediff(string enddate, string startdate)
返回值: int
说明: 返回结束日期减去开始日期的天数。
举例:
hive> select datediff(‘2012-12-08’,‘2012-05-09’) from dual;
213
• 日期增加函数 : date_add
语法: date_add(string startdate, int days)
返回值: string
说明: 返回开始日期startdate增加days天后的日期。
举例:
hive> select date_add(‘2012-12-08’,10) from dual;
2012-12-18
• 日期减少函数 : date_sub
语法: date_sub (string startdate, int days)
返回值: string
说明: 返回开始日期startdate减少days天后的日期。
举例:
hive> select date_sub(‘2012-12-08’,10) from dual;
2012-11-28
3. 条件函数
case when then end
select *,
case
when salary < 5000 then "低等收入"
when salary>= 5000 and salary < 10000 then "中等收入"
when salary > 10000 then "高等收入"
end as level,
case sex
when "female" then 1
when "male" then 0
end as flag
from employee
4.字符串函数: