HIve数仓新零售项目DWS层的构建(Grouping sets)模型

HIve数仓新零售项目

注：大家觉得博客好的话，别忘了点赞收藏呀，本人每周都会更新关于人工智能和大数据相关的内容，内容多为原创，Python Java Scala SQL 代码，CV NLP 推荐系统等，Spark Flink Kafka Hbase Hive Flume等等~写的都是纯干货，各种顶会的论文解读，一起进步。
今天继续和大家分享一下HIve数仓新零售项目
#博学谷IT学习技术支持

---

---

前言

这是一个线下真实HIve数仓的一个搭建项目，还是比较复杂的，主要和大家一起分享一下整个HIve数仓的思路。
整个项目分为：
1.ODS层
2.DWD层
3.DWB层
4.DWS层
5.DM层
6.RPT层
每一层都有每一层的知识点。我会和大家分享从数据源MySQL开始，如何搭建整个完整的项目。

---

一、Grouping sets 模型介绍

一种高效的替代多个UNION ALL语法的模型，个人比较喜欢，非常灵活，速度快。
以下是一个demo案例。
需求：
分别按照月（month）、天（day）、月和天（month,day）统计来访用户userid个数，并获取三者的结果集（一起插入到目标宽表中）。

create table test.t_user(
    month string, 
    day string, 
    userid string
) 
row format delimited fields terminated by ',';

--数据样例
2015-03,2015-03-10,user1
2015-03,2015-03-10,user5
2015-03,2015-03-12,user7
2015-04,2015-04-12,user3
2015-04,2015-04-13,user2
2015-04,2015-04-13,user4
2015-04,2015-04-16,user4
2015-03,2015-03-10,user2
2015-03,2015-03-10,user3
2015-04,2015-04-12,user5
2015-04,2015-04-13,user6
2015-04,2015-04-15,user3
2015-04,2015-04-15,user2
2015-04,2015-04-16,user1

• UNION ALL 写法 ，比较丑而且速度慢，效率低

--3个分组统计而已，简单。统计完再使用union all合并结果集。
--注意union all合并结果集需要各个查询返回字段个数、类型一致，因此需要合理的使用null来填充返回结果。
select month,
       null,
       count(userid)
from test.t_user
group by month

union all

select null,
       day,
       count(userid)
from test.t_user
group by day

union all

select month,
       day,
       count(userid)
from test.t_user
group by month,day;

• grouping sets模型写法
• 根据不同的维度组合进行聚合，等价于将不同维度的GROUP BY结果集进行UNION ALL。

-- Hive的写法
select 
    month,day,count(userid) 
from test.t_user 
    group by month,day 
grouping sets (month,day,(month,day));

-- presto的写法
select
month,
day,
count(*) as cnt
from  test.t_user
group by
grouping sets (month,day,(month,day))

• grouping介绍与使用
• 功能：使用grouping操作来判断当前数据是按照哪个字段来分组的。
• 对于给定的分组，如果分组中包含相应的列，则将位设置为0，否则将其设置为1

select month,
       day,
       count(userid),
       grouping(month)      as m,
       grouping(day)        as d,
       grouping(month, day) as m_d
from test.t_user
group by
   grouping sets (month, day, (month, day));

二、DWS层功能与职责

DWS层: 基于主题统计分析, 此层一般是用于最细粒度的统计操作

• 维度组合:

  日期
  日期+城市
  日期+城市+商圈
  日期+城市+商圈+店铺
  日期+品牌
  日期+大类
  日期+大类+中类
  日期+大类+中列+小类

• 指标:
  销售收入、平台收入、配送成交额、小程序成交额、安卓APP成交额、苹果APP成交额、PC商城成交额、订单量、参 评单量、差评单量、配送单量、退款单量、小程序订单量、安卓APP订单量、苹果APP订单量、PC商城订单量。

三、销售主题统计宽表

最终要求通过group_type来判断指标来自哪个维度的聚合

1.构建目标表

drop database if exists yp_dws
create database if not exists yp_dws;

-- 销售主题日统计宽表
DROP TABLE IF EXISTS yp_dws.dws_sale_daycount;
CREATE TABLE yp_dws.dws_sale_daycount(
   city_id string COMMENT '城市id',
   city_name string COMMENT '城市name',
   trade_area_id string COMMENT '商圈id',
   trade_area_name string COMMENT '商圈名称',
   store_id string COMMENT '店铺的id',
   store_name string COMMENT '店铺名称',
   brand_id string COMMENT '品牌id',
   brand_name string COMMENT '品牌名称',
   max_class_id string COMMENT '商品大类id',
   max_class_name string COMMENT '大类名称',
   mid_class_id string COMMENT '中类id',
   mid_class_name string COMMENT '中类名称',
   min_class_id string COMMENT '小类id',
   min_class_name string COMMENT '小类名称',

   -- 经验字段: 用于标记每一条数据是按照哪个维度计算出来的
   group_type string COMMENT '分组类型：store，trade_area，city，brand，min_class，mid_class，max_class，all',

   --   =======日统计=======
   --   销售收入
   sale_amt DECIMAL(38,2) COMMENT '销售收入',
   --   平台收入
   plat_amt DECIMAL(38,2) COMMENT '平台收入',
   -- 配送成交额
   deliver_sale_amt DECIMAL(38,2) COMMENT '配送成交额',
   -- 小程序成交额
   mini_app_sale_amt DECIMAL(38,2) COMMENT '小程序成交额',
   -- 安卓APP成交额
   android_sale_amt DECIMAL(38,2) COMMENT '安卓APP成交额',
   --  苹果APP成交额
   ios_sale_amt DECIMAL(38,2) COMMENT '苹果APP成交额',
   -- PC商城成交额
   pcweb_sale_amt DECIMAL(38,2) COMMENT 'PC商城成交额',
   -- 成交单量
   order_cnt BIGINT COMMENT '成交单量',
   -- 参评单量
   eva_order_cnt BIGINT COMMENT '参评单量comment=>cmt',
   -- 差评单量
   bad_eva_order_cnt BIGINT COMMENT '差评单量negtive-comment=>ncmt',
   -- 配送成交单量
   deliver_order_cnt BIGINT COMMENT '配送单量',
   -- 退款单量
   refund_order_cnt BIGINT COMMENT '退款单量',
   -- 小程序成交单量
   miniapp_order_cnt BIGINT COMMENT '小程序成交单量',
   -- 安卓APP订单量
   android_order_cnt BIGINT COMMENT '安卓APP订单量',
   -- 苹果APP订单量
   ios_order_cnt BIGINT COMMENT '苹果APP订单量',
   -- PC商城成交单量
   pcweb_order_cnt BIGINT COMMENT 'PC商城成交单量'
)
COMMENT '销售主题日统计宽表'
PARTITIONED BY(dt STRING)
ROW format delimited fields terminated BY '\t'
stored AS orc tblproperties ('orc.compress' = 'SNAPPY');

2.Presto Grouping sets语法实现

insert into yp_dws.dws_sale_daycount
with t0 as (
   select
     -- 列裁剪
     -- 维度字段
     od.dt,
     city_id,
     city_name,
     trade_area_id,
     trade_area_name,
     store_name,
     brand_id,
     brand_name,
     max_class_name,
     max_class_id,
     mid_class_name,
     mid_class_id,
     min_class_name,
     min_class_id,

     -- 指标字段
    order_id,
    order_amount,
    total_price,
    plat_fee,
    delivery_fee,
    order_from,
    evaluation_id,
    geval_scores,
    delievery_id,
    refund_id,
    od.store_id,
    row_number() over (partition by order_id,goods_id ) as rk1, -- 过滤脏数据
    row_number() over (partition by order_id ) as rk2
   from yp_dwb.dwb_order_detail od
     left join  yp_dwb.dwb_shop_detail  sd on od.store_id = sd.id
     left join  yp_dwb.dwb_goods_detail gd on od.goods_id = gd.id
 )
select
    city_id,
    city_name,
    trade_area_id,
    trade_area_name,
    store_id,
    store_name,
    brand_id,
    brand_name,
    max_class_id,
    max_class_name,
    mid_class_id,
    mid_class_name,
    min_class_id,
    min_class_name,
    case when grouping(store_id) = 0      -- if
           then 'store'  -- 日期 + 城市 + 商圈 + 店铺
         when grouping(trade_area_id) = 0  -- else if
           then 'trade_area'  --日期 + 城市 + 商圈
         when grouping(city_id) = 0   -- else if
            then 'city '      --日期 + 城市
         when grouping(brand_id) = 0  -- else if
           then 'brand'  -- 日期 = 品牌
         when grouping(min_class_id) = 0  -- else if
           then 'min_class'  -- 日期 + 大类 + 中类 + 小类
         when grouping(mid_class_id) = 0   -- else if
           then 'mid_class'  -- 日期 + 大类 + 中类
         when grouping(max_class_id) = 0
           then 'max_clas'  -- 日期 + 大类
         else
           'all'  -- 日期
    end as group_type,
      -- 总销售额
    case when grouping(store_id) = 0
            then sum(if(store_id is not null,total_price,0))
         when grouping(trade_area_id) = 0
            then sum(if(trade_area_id is not null,total_price,0))
         when  grouping(city_id) = 0
            then sum(if(city_id is not null,total_price,0))
         when grouping(brand_id) = 0
             then sum(if(brand_id is not null,total_price,0))
         when grouping(min_class_id) = 0
            then sum(if(min_class_id is not null,total_price,0))
        when grouping(mid_class_id) = 0
             then sum(if(mid_class_id is not null,total_price,0))
        when grouping(mid_class_id) = 0
             then sum(if(mid_class_id is not null,total_price,0))   -- 聚合定制
        when grouping(max_class_id) = 0
             then sum(if(max_class_id is not null,total_price,0))
        else
            sum(if(dt is not null,total_price,0))  -- 日期
    end as sale_amt,
    -- 平台收入
       case when grouping(store_id) = 0
            then sum(if(store_id is not null,plat_fee,0))
         when grouping(trade_area_id) = 0
            then sum(if(trade_area_id is not null,plat_fee,0))
         when  grouping(city_id) = 0
            then sum(if(city_id is not null,plat_fee,0))
         when grouping(brand_id) = 0
             then sum(if(brand_id is not null,plat_fee,0))
         when grouping(min_class_id) = 0
            then sum(if(min_class_id is not null,plat_fee,0))
        when grouping(mid_class_id) = 0
             then sum(if(mid_class_id is not null,plat_fee,0))
        when grouping(mid_class_id) = 0
             then sum(if(mid_class_id is not null,plat_fee,0))   -- 聚合定制
        when grouping(max_class_id) = 0
             then sum(if(max_class_id is not null,plat_fee,0))
        else
            sum(if(dt is not null,plat_fee,0))  -- 日期
    end as plat_amt,
     -- 配送成交额
       case when grouping(store_id) = 0
            then sum(if(store_id is not null and delievery_id is not null,total_price,0))
         when grouping(trade_area_id) = 0
            then sum(if(trade_area_id is not null  and delievery_id is not null,total_price,0))
         when  grouping(city_id) = 0
            then sum(if(city_id is not null  and delievery_id is not null,total_price,0))
         when grouping(brand_id) = 0
             then sum(if(brand_id is not null  and delievery_id is not null,total_price,0))
         when grouping(min_class_id) = 0
            then sum(if(min_class_id is not null  and delievery_id is not null,total_price,0))
        when grouping(mid_class_id) = 0
             then sum(if(mid_class_id is not null  and delievery_id is not null,total_price,0))
        when grouping(mid_class_id) = 0
             then sum(if(mid_class_id is not null  and delievery_id is not null,total_price,0))   -- 聚合定制
        when grouping(max_class_id) = 0
             then sum(if(max_class_id is not null  and delievery_id is not null,total_price,0))
        else
            sum(if(dt is not null  and delievery_id is not null,total_price,0))  -- 日期
    end as deliver_sale_amt,
     -- 小程序成交额
       case when grouping(store_id) = 0
            then sum(if(store_id is not null and order_from = 'miniapp',total_price,0))
         when grouping(trade_area_id) = 0
            then sum(if(trade_area_id is not null and order_from = 'miniapp',total_price,0))
         when  grouping(city_id) = 0
            then sum(if(city_id is not null  and order_from = 'miniapp',total_price,0))
         when grouping(brand_id) = 0
             then sum(if(brand_id is not null  and order_from = 'miniapp',total_price,0))
         when grouping(min_class_id) = 0
            then sum(if(min_class_id is not null  and order_from = 'miniapp',total_price,0))
        when grouping(mid_class_id) = 0
             then sum(if(mid_class_id is not null  and order_from = 'miniapp',total_price,0))
        when grouping(mid_class_id) = 0
             then sum(if(mid_class_id is not null  and order_from = 'miniapp',total_price,0))   -- 聚合定制
        when grouping(max_class_id) = 0
             then sum(if(max_class_id is not null  and order_from = 'miniapp',total_price,0))
        else
            sum(if(dt is not null  and order_from = 'miniapp',total_price,0))  -- 日期
    end as mini_app_sale_amt,
     -- android成交额
       case when grouping(store_id) = 0
            then sum(if(store_id is not null and order_from = 'android',total_price,0))
         when grouping(trade_area_id) = 0
            then sum(if(trade_area_id is not null and order_from = 'android',total_price,0))
         when  grouping(city_id) = 0
            then sum(if(city_id is not null  and order_from = 'android',total_price,0))
         when grouping(brand_id) = 0
             then sum(if(brand_id is not null  and order_from = 'android',total_price,0))
         when grouping(min_class_id) = 0
            then sum(if(min_class_id is not null  and order_from = 'android',total_price,0))
        when grouping(mid_class_id) = 0
             then sum(if(mid_class_id is not null  and order_from = 'android',total_price,0))
        when grouping(mid_class_id) = 0
             then sum(if(mid_class_id is not null  and order_from = 'android',total_price,0))   -- 聚合定制
        when grouping(max_class_id) = 0
             then sum(if(max_class_id is not null  and order_from = 'android',total_price,0))
        else
            sum(if(dt is not null  and order_from = 'android',total_price,0))  -- 日期
    end as android_sale_amt,
      -- ios成交额
    case when grouping(store_id) = 0
            then sum(if(store_id is not null and order_from = 'ios',total_price,0))
         when grouping(trade_area_id) = 0
            then sum(if(trade_area_id is not null and order_from = 'ios',total_price,0))
         when  grouping(city_id) = 0
            then sum(if(city_id is not null  and order_from = 'ios',total_price,0))
         when grouping(brand_id) = 0
             then sum(if(brand_id is not null  and order_from = 'ios',total_price,0))
         when grouping(min_class_id) = 0
            then sum(if(min_class_id is not null  and order_from = 'ios',total_price,0))
        when grouping(mid_class_id) = 0
             then sum(if(mid_class_id is not null  and order_from = 'ios',total_price,0))
        when grouping(mid_class_id) = 0
             then sum(if(mid_class_id is not null  and order_from = 'ios',total_price,0))   -- 聚合定制
        when grouping(max_class_id) = 0
             then sum(if(max_class_id is not null  and order_from = 'ios',total_price,0))
        else
            sum(if(dt is not null  and order_from = 'ios',total_price,0))  -- 日期
    end as ios_sale_amt,
       -- pcweb成交额
    case when grouping(store_id) = 0
            then sum(if(store_id is not null and order_from = 'pcweb',total_price,0))
         when grouping(trade_area_id) = 0
            then sum(if(trade_area_id is not null and order_from = 'pcweb',total_price,0))
         when  grouping(city_id) = 0
            then sum(if(city_id is not null  and order_from = 'pcweb',total_price,0))
         when grouping(brand_id) = 0
             then sum(if(brand_id is not null  and order_from = 'pcweb',total_price,0))
         when grouping(min_class_id) = 0
            then sum(if(min_class_id is not null  and order_from = 'pcweb',total_price,0))
        when grouping(mid_class_id) = 0
             then sum(if(mid_class_id is not null  and order_from = 'pcweb',total_price,0))
        when grouping(mid_class_id) = 0
             then sum(if(mid_class_id is not null  and order_from = 'pcweb',total_price,0))   -- 聚合定制
        when grouping(max_class_id) = 0
             then sum(if(max_class_id is not null  and order_from = 'pcweb',total_price,0))
        else
            sum(if(dt is not null  and order_from = 'pcweb',total_price,0))  -- 日期
    end as pcweb_sale_amt,
    -- 成交单量
    case when grouping(store_id) = 0
            then count(if(store_id is not null and rk2 = 1,order_id,null))
         when grouping(trade_area_id) = 0
            then count(if(trade_area_id is not null and rk2 = 1,order_id,null))
         when  grouping(city_id) = 0
            then count(if(city_id is not null and rk2=1,order_id,null))
         when grouping(brand_id) = 0
             then count(if(brand_id is not null and rk2=1,order_id,null))
         when grouping(min_class_id) = 0
            then count(if(min_class_id is not null and rk2=1,order_id,null))
        when grouping(mid_class_id) = 0
             then count(if(mid_class_id is not null and rk2=1,order_id,null))
        when grouping(mid_class_id) = 0
             then count(if(mid_class_id is not null and rk2=1,order_id,null))   -- 聚合定制
        when grouping(max_class_id) = 0
             then count(if(max_class_id is not null and rk2=1,order_id,null))
        else
            count(if(dt is not null and rk2=1,order_id,null))  -- 日期
    end as order_cnt,
     -- 参评单量
    case when grouping(store_id) = 0
            then count(if(store_id is not null and rk2=1 and evaluation_id is not null and evaluation_id is not null,order_id,null))
         when grouping(trade_area_id) = 0
            then count(if(trade_area_id is not null and rk2=1 and evaluation_id is not null,order_id,null))
         when  grouping(city_id) = 0
            then count(if(city_id is not null and rk2=1 and evaluation_id is not null,order_id,null))
         when grouping(brand_id) = 0
             then count(if(brand_id is not null and rk2=1 and evaluation_id is not null,order_id,null))
         when grouping(min_class_id) = 0
            then count(if(min_class_id is not null and rk2=1 and evaluation_id is not null,order_id,null))
        when grouping(mid_class_id) = 0
             then count(if(mid_class_id is not null and rk2=1 and evaluation_id is not null,order_id,null))
        when grouping(mid_class_id) = 0
             then count(if(mid_class_id is not null and rk2=1 and evaluation_id is not null,order_id,null))   -- 聚合定制
        when grouping(max_class_id) = 0
             then count(if(max_class_id is not null and rk2=1 and evaluation_id is not null,order_id,null))
        else
            count(if(dt is not null and rk2=1 and evaluation_id is not null,order_id,null))  -- 日期
    end as eva_order_cnt,

     -- 差评单量
    case when grouping(store_id) = 0
            then count(if(store_id is not null and rk2=1 and evaluation_id is not null and geval_scores <= 6,order_id,null))
         when grouping(trade_area_id) = 0
            then count(if(trade_area_id is not null and rk2=1 and evaluation_id is not null and geval_scores <= 6,order_id,null))
         when  grouping(city_id) = 0
            then count(if(city_id is not null and rk2=1 and evaluation_id is not null and geval_scores <= 6,order_id,null))
         when grouping(brand_id) = 0
             then count(if(brand_id is not null and rk2=1 and evaluation_id is not null and geval_scores <= 6,order_id,null))
         when grouping(min_class_id) = 0
            then count(if(min_class_id is not null and rk2=1 and evaluation_id is not null and geval_scores <= 6,order_id,null))
        when grouping(mid_class_id) = 0
             then count(if(mid_class_id is not null and rk2=1 and evaluation_id is not null and geval_scores <= 6,order_id,null))
        when grouping(mid_class_id) = 0
             then count(if(mid_class_id is not null and rk2=1 and evaluation_id is not null and geval_scores <= 6,order_id,null))   -- 聚合定制
        when grouping(max_class_id) = 0
             then count(if(max_class_id is not null and rk2=1 and evaluation_id is not null and geval_scores <= 6,order_id,null))
        else
            count(if(dt is not null and rk2=1 and evaluation_id is not null and geval_scores <= 6,order_id,null))  -- 日期
    end as bad_eva_order_cnt,
    -- 配送单量
    case when grouping(store_id) = 0
            then count(if(store_id is not null and rk2=1 and delievery_id is not null,order_id,null))
         when grouping(trade_area_id) = 0
            then count(if(trade_area_id is not null and rk2=1 and delievery_id is not null,order_id,null))
         when  grouping(city_id) = 0
            then count(if(city_id is not null and rk2=1 and delievery_id is not null,order_id,null))
         when grouping(brand_id) = 0
             then count(if(brand_id is not null and rk2=1 and delievery_id is not null,order_id,null))
         when grouping(min_class_id) = 0
            then count(if(min_class_id is not null and rk2=1 and delievery_id is not null,order_id,null))
        when grouping(mid_class_id) = 0
             then count(if(mid_class_id is not null and rk2=1 and delievery_id is not null,order_id,null))
        when grouping(mid_class_id) = 0
             then count(if(mid_class_id is not null and rk2=1 and delievery_id is not null,order_id,null))   -- 聚合定制
        when grouping(max_class_id) = 0
             then count(if(max_class_id is not null and rk2=1 and delievery_id is not null,order_id,null))
        else
            count(if(dt is not null and rk2=1 and delievery_id is not null,order_id,null))  -- 日期
    end as deliver_order_cnt,
    -- 退款单量
    case when grouping(store_id) = 0
            then count(if(store_id is not null and rk2=1 and refund_id is not null,order_id,null))
         when grouping(trade_area_id) = 0
            then count(if(trade_area_id is not null and rk2=1 and refund_id is not null,order_id,null))
         when  grouping(city_id) = 0
            then count(if(city_id is not null and rk2=1 and refund_id is not null,order_id,null))
         when grouping(brand_id) = 0
             then count(if(brand_id is not null and rk2=1 and refund_id is not null,order_id,null))
         when grouping(min_class_id) = 0
            then count(if(min_class_id is not null and rk2=1 and refund_id is not null,order_id,null))
        when grouping(mid_class_id) = 0
             then count(if(mid_class_id is not null and rk2=1 and refund_id is not null,order_id,null))
        when grouping(mid_class_id) = 0
             then count(if(mid_class_id is not null and rk2=1 and refund_id is not null,order_id,null))   -- 聚合定制
        when grouping(max_class_id) = 0
             then count(if(max_class_id is not null and rk2=1 and refund_id is not null,order_id,null))
        else
            count(if(dt is not null and rk2=1 and refund_id is not null,order_id,null))  -- 日期
    end as refund_order_cnt,
    -- 小程序成交单量
    case when grouping(store_id) = 0
            then count(if(store_id is not null and rk2=1 and order_from = 'miniapp',order_id,null))
         when grouping(trade_area_id) = 0
            then count(if(trade_area_id is not null and rk2=1 and order_from = 'miniapp',order_id,null))
         when  grouping(city_id) = 0
            then count(if(city_id is not null and rk2=1  and order_from = 'miniapp',order_id,null))
         when grouping(brand_id) = 0
             then count(if(brand_id is not null and rk2=1  and order_from = 'miniapp',order_id,null))
         when grouping(min_class_id) = 0
            then count(if(min_class_id is not null and rk2=1  and order_from = 'miniapp',order_id,null))
        when grouping(mid_class_id) = 0
             then count(if(mid_class_id is not null and rk2=1  and order_from = 'miniapp',order_id,null))
        when grouping(mid_class_id) = 0
             then count(if(mid_class_id is not null and rk2=1  and order_from = 'miniapp',order_id,null))   -- 聚合定制
        when grouping(max_class_id) = 0
             then count(if(max_class_id is not null and rk2=1  and order_from = 'miniapp',order_id,null))
        else
            count(if(dt is not null  and rk2=1 and order_from = 'miniapp',order_id,null))  -- 日期
    end as miniapp_order_cnt,
       -- android成交单量
    case when grouping(store_id) = 0
            then count(if(store_id is not null and rk2=1 and order_from = 'android',order_id,null))
         when grouping(trade_area_id) = 0
            then count(if(trade_area_id is not  null and rk2=1 and order_from = 'android',order_id,null))
         when  grouping(city_id) = 0
            then count(if(city_id is not null and rk2=1 and order_from = 'android',order_id,null))
         when grouping(brand_id) = 0
             then count(if(brand_id is not null and rk2=1 and order_from = 'android',order_id,null))
         when grouping(min_class_id) = 0
            then count(if(min_class_id is not null and rk2=1  and order_from = 'android',order_id,null))
        when grouping(mid_class_id) = 0
             then count(if(mid_class_id is not null and rk2=1 and order_from = 'android',order_id,null))
        when grouping(mid_class_id) = 0
             then count(if(mid_class_id is not null and rk2=1 and order_from = 'android',order_id,null))   -- 聚合定制
        when grouping(max_class_id) = 0
             then count(if(max_class_id is not null and rk2=1  and order_from = 'android',order_id,null))
        else
            count(if(dt is not null and rk2=1 and order_from = 'android',order_id,null))  -- 日期
    end as android_order_cnt,
    -- ios成交单量
    case when grouping(store_id) = 0
            then count(if(store_id is not null and rk2=1  and order_from = 'ios',order_id,null))
         when grouping(trade_area_id) = 0
            then count(if(trade_area_id is not null and rk2=1  and order_from = 'ios',order_id,null))
         when  grouping(city_id) = 0
            then count(if(city_id is not null and rk2=1   and order_from = 'ios',order_id,null))
         when grouping(brand_id) = 0
             then count(if(brand_id is not null and rk2=1   and order_from = 'ios',order_id,null))
         when grouping(min_class_id) = 0
            then count(if(min_class_id is not null and rk2=1   and order_from = 'ios',order_id,null))
        when grouping(mid_class_id) = 0
             then count(if(mid_class_id is not null and rk2=1   and order_from = 'ios',order_id,null))
        when grouping(mid_class_id) = 0
             then count(if(mid_class_id is not null and rk2=1   and order_from = 'ios',order_id,null))   -- 聚合定制
        when grouping(max_class_id) = 0
             then count(if(max_class_id is not null and rk2=1   and order_from = 'ios',order_id,null))
        else
            count(if(dt is not null and rk2=1   and order_from = 'ios',order_id,null))  -- 日期
    end as ios_order_cnt,
      -- pcweb成交单量
    case when grouping(store_id) = 0
            then count(if(store_id is not null and rk2=1  and order_from = 'pcweb',order_id,null))
         when grouping(trade_area_id) = 0
            then count(if(trade_area_id is not null and rk2=1  and order_from = 'pcweb',order_id,null))
         when  grouping(city_id) = 0
            then count(if(city_id is not null and rk2=1   and order_from = 'pcweb',order_id,null))
         when grouping(brand_id) = 0
             then count(if(brand_id is not null and rk2=1   and order_from = 'pcweb',order_id,null))
         when grouping(min_class_id) = 0
            then count(if(min_class_id is not null and rk2=1   and order_from = 'pcweb',order_id,null))
        when grouping(mid_class_id) = 0
             then count(if(mid_class_id is not null and rk2=1   and order_from = 'pcweb',order_id,null))
        when grouping(mid_class_id) = 0
             then count(if(mid_class_id is not null and rk2=1   and order_from = 'pcweb',order_id,null))   -- 聚合定制
        when grouping(max_class_id) = 0
             then count(if(max_class_id is not null and rk2=1   and order_from = 'pcweb',order_id,null))
        else
            count(if(dt is not null  and order_from = 'pcweb',order_id,null))  -- 日期
    end as pcweb_order_cnt,
    dt
from t0
where rk1 = 1
group by
grouping sets (
  dt,
 (dt,city_id,city_name),
 (dt,city_id,city_name,trade_area_id,trade_area_name),
 (dt,city_id,city_name,trade_area_id,trade_area_name,store_id,store_name),
 (dt,brand_id,brand_name),
 (dt,max_class_id,max_class_name),
 (dt,max_class_id,max_class_name,mid_class_id,mid_class_name),
 (dt,max_class_id,max_class_name,mid_class_id,mid_class_name,min_class_id,min_class_name)
);

这里主要是运用了grouping和grouping sets的语法，如果不了解可以百度一下。

---

总结

这里介绍了HIve数仓新零售项目DWS层的构建(Grouping sets)模型，Grouping sets模型适合于多维度，多指标的稀疏宽表的构建，可以把不同的维度放在同一张宽表中，方便以后查询。同时在建立聚合字段的时候，可以根据每个维度进行定制聚合的操作。比较灵活。
如果对grouping和grouping sets的语法有疑问，可以留言讨论。