python实现sql中的lag_Hive窗口函数之LAG、LEAD、FIRST_VALUE、LAST_VALUE的用法

一、创建表：

create table windows_ss

(

polno string,

eff_date string,

userno string

)

ROW FORMAT DELIMITED

FIELDS TERMINATED BY ‘,‘

stored as textfile;

数据准备：

P066666666666,2016-04-02 09:00:02,user01

P066666666666,2016-04-02 09:00:00,user02

P066666666666,2016-04-02 09:03:04,user11

P066666666666,2016-04-02 09:50:05,user03

P066666666666,2016-04-02 10:00:00,user51

P066666666666,2016-04-02 09:10:00,user09

P066666666666,2016-04-02 09:50:01,user32

P088888888888,2016-04-02 09:00:02,user41

P088888888888,2016-04-02 09:00:00,user55

P088888888888,2016-04-02 09:03:04,user23

P088888888888,2016-04-02 09:50:05,user80

P088888888888,2016-04-02 10:00:00,user08

P088888888888,2016-04-02 09:10:00,user22

P088888888888,2016-04-02 09:50:01,user31

将数据导入Hive表中：

LOAD DATA LOCAL INPATH  ‘/home/hadoop/testhivedata/windows_ss.txt‘  OVERWRITE INTO TABLE windows_ss;

LAG

LAG(col,n,DEFAULT) 用于统计窗口内往上第n行值

第一个参数为列名，第二个参数为往上第n行(可选，默认为1)，第三个参数为默认值(当往上第n行为NULL时候，取默认值，如不指定，则为NULL)

SELECT

polno,

eff_date,

userno,

ROW_NUMBER() OVER(PARTITION BY polno ORDER BY eff_date) AS rn,

LAG(eff_date,1,‘1970-01-01 00:00:00‘) OVER(PARTITION BY polno ORDER BY eff_date) AS last_1_time,

LAG(eff_date,2) OVER(PARTITION BY polno ORDER BY eff_date) AS last_2_time

FROM windows_ss;

结果：

polno                        eff_date                              userno       rn    last_1_time                  last_2_time

P066666666666     2016-04-02 09:00:00      user02     1     1970-01-01 00:00:00      NULL

P066666666666     2016-04-02 09:00:02      user01     2     2016-04-02 09:00:00      NULL

P066666666666     2016-04-02 09:03:04      user11     3     2016-04-02 09:00:02      2016-04-02 09:00:00

P066666666666     2016-04-02 09:10:00      user09     4     2016-04-02 09:03:04      2016-04-02 09:00:02

P066666666666     2016-04-02 09:50:01      user32     5     2016-04-02 09:10:00      2016-04-02 09:03:04

P066666666666     2016-04-02 09:50:05      user03     6     2016-04-02 09:50:01      2016-04-02 09:10:00

P066666666666     2016-04-02 10:00:00      user51     7     2016-04-02 09:50:05      2016-04-02 09:50:01

P088888888888     2016-04-02 09:00:00      user55     1     1970-01-01 00:00:00      NULL

P088888888888     2016-04-02 09:00