【黑马甄选离线数仓day06_核销主题域开发】

1. 核销主题_DWD和DWM层

1.0 ODS层

操作数据存储层: ==Operate Data Store==

核心理念: 几乎和源数据保持一致,粒度相同

注意事项: ==同步方式(全量同步,全量覆盖,增量仅新增,增量新增和更新)+ 内部表 + 分区表(部分) + 指定字符分隔符 +orc + zlib==

第二天的时候已经完成了从mysql以及sqlserver抽取数据到ods层的过程

1.1 DWD层

数据仓库细节层: ==Data Warehouse Detail==

DWM层提前合并部分数据

注意事项: ==同步方式(全量同步,全量覆盖,增量仅新增,增量新增和更新)+ 内部表 + 分区表(部分) + 指定字符分隔符 +orc + snappy==

核心理念:

清洗 转换

区分事实表和维度表

降维拉宽

1- 将相关的表进行合并操作:  
     事实表中多个事实表之间有关联, 一般是必须合并的
     事实表和维度表合并操作: 和有关联的维度表进行合并
     
2- 如果事实表和其他的表没有关联, 此时尝试和维度表合并即可
     注意: 如果维度表都没有, 无需合并, 进行清洗过滤即可
     
3- 设计拉宽表:
    - a)原始表中重要信息(不好判断的时候,尽量保留所有字段即可)
    - b)一些必要类型字段(比如说is_day_clear,is_balance_consume)
    - c)维度拉宽(比如对日期/时间进行处理、门店信息拉宽、商品信息拉宽)
    - d)还要注意,保持数仓中字段名的统一(类型名、指标、维度字段等)

本次DWD层,共设计为12张事实表,在添加数据的时候,注意具体计算只计算增量,全量计算直接去掉where条件即可。或者在调度时使用补数的方法,将历史分区一天一天跑出来即可!

门店销售明细宽表(8张表降维拉宽): dwd_sale_store_sale_dtl_i
​
线上余额支付明细表: dwd_sale_shop_sale_balance_pay_i
商城订单表(核销表):    dwd_sold_shop_order_i
商城订单明细表(核销表): dwd_sold_shop_order_item_i
​
商城订单核销明细宽表(10张表降维拉宽): dwd_sold_shop_order_dtl_i
​
门店库调表(3张表降维拉宽): dwd_stock_store_stock_adj_i
​
门店收货表(3张表降维拉宽):dwd_order_store_receive_i
门店退货表(3张表降维拉宽): dwd_order_store_return_to_vendor_i
门店退配表(3张表降维拉宽): dwd_order_store_return_to_dc_i
​
门店调入表(3张表降维拉宽):dwd_order_store_alloc_in_i
门店调出表(3张表降维拉宽):dwd_order_store_alloc_out_i
​
门店要货表(3张表降维拉宽):dwd_order_store_require_i

1.1.1 门店销售明细宽表

此表为常见的明细打宽的开发任务, 整个业务过程是门店下单, 粒度为订单明细, 维度包含时间、门店、商品、品类等, 事实包括销售数量、金额、成本、余额支付金额等

本次, 我们主要将门店销售的核心三张表进行拉宽操作, 包括门店销售信息表、门店销售明细表、门店销售支付表进行合并拉宽, 同时还需要带上相关的维度表

涉及事实表: 3张
ods_sale_store_sale_info_i(门店销售信息表): 主要描述了门店线下的的销售明细,包括单号、时间、金额、渠道、会员等信息。
ods_sale_store_sale_dtl_i(门店销售明细表): 相对于销售信息表, 多了具体的商品信息, 库存信息等
    注意:
        1)因为组合商品和组合成分都是会记录到明细表的,这里只需要分析组合商品,所以需要排除掉组合成分。
            比如牙膏和牙刷一起组合销售,则牙膏和牙刷一起为组合商品,牙膏、牙刷为组合成分。
            combination_flag int comment '1-组合商品,2-组合成分'
        2) 有时候库存调整要做一些冲减单,这些单据不是正常的销售单,所以需要排除。过滤条件为offset_flag = 0  
ods_sale_store_sale_pay_i(门店销售支付表): 主要描述了每一单的支付类别、支付金额等信息。这个表只是为了取余额销售的信息,所以这里让 pay_type_id=’201’
​
三张表的关联条件是order_no相同。订单表和明细表进行inner join,支付表要跟订单表做左关联(因为不是每一单都是余额支付)
​
涉及维度表: 5张
​
dwd_dim_date_f(时间维度表):记录了一个日期对应的不同的日期,如周一日期、周末日期、月一日期、月末日期等。
dwd_dim_source_type_map_i(交易类型映射表):记录了原始交易渠道对应的归类情况。
dwd_dim_store_i(门店表):记录了门店的基本信息,包括门店销售类型、分店类型、城市、区域、人数、状态等信息。
dwd_dim_goods_i(商品表):记录商品的基本信息,包括ID,编码,名称,分类,售卖信息,订货信息等。
dwd_dim_store_goods_i(门店商品表):跟商品表不同的是,商品表记录的是商品通用的信息,而门店商品表里会记录一些门店独有的信息,比如日清信息,采购信息,柜组信息等。

【黑马甄选离线数仓day06_核销主题域开发】_第1张图片

建表语句
CREATE TABLE IF NOT EXISTS dwd.dwd_sale_store_sale_dtl_i(
    -- 门店销售信息表
    trade_date_time         STRING COMMENT '销售时间',
    trade_date              STRING COMMENT '交易日期',
    -- 时间维度表
    week_trade_date         STRING COMMENT '周一日期',
    month_trade_date        STRING COMMENT '月一日期',
    -- 门店销售信息表 
    hourly                  BIGINT COMMENT '交易小时(0-23)',
    quarter                 BIGINT COMMENT '刻钟:1.0-15,2.15-30,3.30-45,4.45-60',
    quarters                BIGINT COMMENT '刻钟数:hourly*4+quarters',
    parent_store_no         STRING COMMENT '母店编码',
    store_no                STRING COMMENT '店铺编码',
    -- 门店表
    store_name              STRING COMMENT '店铺名称',
    store_sale_type         BIGINT COMMENT '店铺销售类型',
    store_type_code         BIGINT COMMENT '分店类型',
    worker_num              BIGINT COMMENT '员工人数',
    store_area              DECIMAL(27, 2) COMMENT '门店面积',
    city_id                 BIGINT COMMENT '城市ID',
    city_name               STRING COMMENT '城市名称',
    region_code             STRING COMMENT '区域编码',
    region_name             STRING COMMENT '区域名称',
    is_day_clear            BIGINT COMMENT '是否日清:0否,1是',
    -- 门店销售信息表
    trade_type              BIGINT COMMENT '结算类型(0.正常交易,1.赠品发放,2.退货,4.培训,5.取消交易)',
    -- 交易类型映射表
    source_type             BIGINT COMMENT '交易来源1:线下POS;2:三方平台;3:传智鲜商城;4:黑马优选团;5:传智大客户;6:传智其他;7:黑马优选;8:优选海淘;9:优选大客户;10:优选POS;11:优选APP;12:优选H5;13:店长工具线下;14:店长工具线上;15:黑马其他',
    source_type_name        STRING COMMENT '交易来源名称',
    -- 门店销售信息表
    sale_type               BIGINT COMMENT '销售类型 1.实物,2.代客,3.优选小程序,4.离店,5.传智鲜小程序,6.第三方平台,7.其他,8.大客户',
    member_type             BIGINT COMMENT '会员类型:0非会员,1线上会员,2实体卡会员',
    is_balance_consume      BIGINT COMMENT '是否有余额支付:0否,1是',
    parent_order_no         STRING COMMENT '母订单编号',
    order_no                STRING COMMENT '订单编号',
    pos_no                  STRING COMMENT 'Pos机号',
    ser_id                  STRING COMMENT 'POS机当天序号从1开始递增',
    item                    BIGINT COMMENT '商品在小票的位置',
    `sort`                  BIGINT COMMENT '组合商品分割商品拆出位置',
    pay_time                STRING COMMENT '支付时间',
    last_update_time        STRING COMMENT '最后更新时间',
    cashier_no              STRING COMMENT '收银员编码',
    cashier_name            STRING COMMENT '收银员名称',
    share_user_id           STRING COMMENT '分享人用户ID',
    commission_amount       DECIMAL(27, 2) COMMENT '佣金',
​
    zt_id                   BIGINT COMMENT '中台ID',
    member_id               BIGINT COMMENT '会员ID',
    card_no                 STRING COMMENT '卡号',
    -- 商品表
    first_category_no       STRING COMMENT '一级分类编码',
    first_category_name     STRING COMMENT '一级分类名称',
    second_category_no      STRING COMMENT '二级分类编码',
    second_category_name    STRING COMMENT '二级分类名称',
    third_category_no       STRING COMMENT '三级分类编码',
    third_category_name     STRING COMMENT '三级分类名称',
    -- 门店销售明细表
    goods_no                STRING COMMENT '商品编码',
    -- 商品表
    goods_name              STRING COMMENT '商品名称',
    spec                    STRING COMMENT '单位',
    -- 门店销售明细表
    is_component            BIGINT COMMENT '是否为组合商品:0否,1是',
    -- 门店商品表
    supply_team             BIGINT COMMENT '供应链团队 1.平台商品,2.优选标品,3.传智鲜标品,4.传智鲜生鲜,5优选POS商品',
    dc_no                   STRING COMMENT '采购仓库编号',
    dc_name                 STRING COMMENT '采购仓库名称',
    group_no                STRING COMMENT '采购柜组编号',
    group_name              STRING COMMENT '采购柜组名称',
    -- 门店销售明细表
    trade_mode_id           BIGINT COMMENT '结算方式:1购销,2联营',
    vendor_id               BIGINT COMMENT '供应商ID',
    contract_no             STRING COMMENT '合同编号',
    -- 门店商品表
    is_clean                BIGINT COMMENT '商品是否日清:0否,1是',
    -- 门店销售明细表
    is_daily_clear          BIGINT COMMENT '商品是否参加日清活动:0否,1是',
    sale_qty                DECIMAL(27, 3) COMMENT '商品销售数量',
    sale_amount             DECIMAL(27, 2) COMMENT '商品销售金额',
    dis_amount              DECIMAL(27, 2) COMMENT '商品折扣金额',
    sale_cost               DECIMAL(27, 2) COMMENT '商品销售成本',
    -- 门店销售支付表
    balance_amount          DECIMAL(27, 2) COMMENT '余额支付',
    -- 当前写入时间: current_timestamp
    write_time              TIMESTAMP COMMENT '写入时间'
)
COMMENT '门店销售明细表'
partitioned by(dt STRING COMMENT '核销日期')
row format delimited fields terminated by ','
stored as orc
tblproperties ('orc.compress'='SNAPPY');
需求梳理

【黑马甄选离线数仓day06_核销主题域开发】_第2张图片

  • 先将三个核心事实表进行合并操作, 形成结果数据表

销售日期:基于门店销售信息表中的库存日期处理
注意:所有时间数据都建议格式化下
母店编码和店铺编码: 母店编码为销售信息表的门店编码, 门店编码使用销售信息表的销售门店编码,如果没有, 再使用门店编码
交易来源:如果为null, 返回1,代表线下POS
会员类型: 如果以OL- 或者 SF- 开头的为1线上会员, 如果是''或者null为0非会员, 否则为实体卡会员
是否余额支付: 基于门店销售支付表,如果订单id为null返回0代表不是,否则返回1代表是
结算类型: 判断trade_id,如果为1 返回0(正常交易), 为2返回2, 如果为3,4返回5
母订单编号: 判断明细表中商品销售金额sale_amount如果小于0采用销售表的source_order_sn(退款单据ID),反之如果大于等于0采用销售表的parent_order_sn. 否则选order_no
item,sort,cashier_no,cashier_name,zt_id(member_center_sn),member_id: 如果为null设为0
支付时间: 选择商品信息表的支付日期pay_date
最后更新时间: 选择商品信息表的库存时间deal_date
是否为组合类型: 判断明细表combination_flag,如果为1返回1(是),否则返回0(否)
trade_mode_id,share_user_id,commission_amount: 如果为null, 设置为0
余额支付金额: 判断单据总支付金额如果等于0 , 则结果为0; 如果余额支付金额为null,则结果也为0,                                          否则使用余额支付金额 * (商品销售金额 / 单据总支付金额)

3个表关联字段

trade_date_time,
trade_date,
hourly,
minute,
parent_store_no,
store_no,
source_type,
member_type,
is_balance_consume,
trade_type,
parent_order_no,
order_no,
pos_no,
ser_id,item,
`sort`,
pay_time,
last_update_time,
cashier_no,
cashier_name,
zt_id,
member_id,
card_no,
goods_no,
is_component,
trade_mode_id,
vendor_id,
contract_no,
is_daily_clear,
share_user_id,
commission_amount,
sale_qty,
sale_amount,
dis_amount,
sale_cost,
balance_amount

代码实现:

select
    date_format(t2.deal_date,'yyyy-MM-dd HH:mm:ss') as trade_date_time,
    date_format(t2.trade_date,'yyyy-MM-dd HH:mm:ss') as trade_date ,
    hour(t2.deal_date) as hourly,
    minute(t2.deal_date) as minute,
    t2.store_no as parent_store_no,
    coalesce(t2.sale_store_no,t2.store_no) as store_no,
    coalesce(t2.source_type,1) as source_type,
    if(
        t2.member_type like 'OL-%' OR t2.member_type like 'SF-%',
        1,
        if(
            t2.member_type = '' OR t2.member_type is null,
            0,
            2
        )
    ) as member_type,
    if(
        t3.order_no is null,
        0,
        1
    ) as is_balance_consume,
    if(
        t2.trade_id = 1,
        0,
        if(
            t2.trade_id = 2,
            2,
            5
        )
    ) as trade_type,
    if(
        t1.sale_amount < 0,
        t2.source_order_sn,
        if(
            t1.sale_amount >= 0,
            t2.parent_order_sn,
            t1.order_no
        )
    ) as parent_order_no,
   t2.order_no,
   t2.pos_no,
   t2.ser_id,
   coalesce(t1.item,0) as item,
   coalesce(t1.`sort`,0) as sort,
   date_format(t2.pay_date,'yyyy-MM-dd HH:mm:ss') as pay_time,
   date_format(t2.deal_date,'yyyy-MM-dd HH:mm:ss') as last_update_time,
   coalesce(t2.cashier_no,0)  as cashier_no,
   coalesce(t2.cashier_name,0) as cashier_name,
   coalesce(t2.member_center_sn,0) as zt_id,
   coalesce(t2.member_id,0) as member_id,
   t2.card_no,
   t1.goods_no,
    if(
        t1.combination_flag = 1,
        1,
        0
    ) as is_component,
   coalesce(t1.trade_mode_id,0) as trade_mode_id,
    t1.vendor_id,
    t1.contract_no,
    t1.is_daily_clear,
    coalesce(t1.share_user_id,0) as share_user_id,
    coalesce(t1.commission_amount,0) as commission_amount,
    t1.sale_qty,
    t1.sale_amount,
    t1.dis_amount,
    t1.sale_cost,
    if(
        t2.total_pay_amount = 0,
        0,
        if(
            t3.pay_amount is null,
            0,
            cast(t3.pay_amount * t1.sale_amount / t2.total_pay_amount as decimal(27,2))
        )
    )as balance_amount

from (select * from ods.ods_sale_store_sale_dtl_i where combination_flag != 2 and offset_flag = 0) t1
    join ods.ods_sale_store_sale_info_i t2 on t2.order_no = t1.order_no
    left join (select * from ods.ods_sale_store_sale_pay_i where pay_type_id = '201') t3 on t2.order_no = t3.order_no ;
  • 接着基于结果和其他五个维度表关联, 生成最终维度表

刻钟: 判断0-14为1,15-29为2,30-44为3,45-59位4
刻钟数: 计算格式为hourly*4+刻钟
销售类型: 判断source_type或者original_source_type,当为1返回1, 当为9返回5 , 当为 4,5,6,7,8返回6. 当为11返回8 ,否则返回7
供应链类型: 判断门店商品信息表的 tag 商品标识为null, 返回4
采购仓库编号:	判断门店商品信息表的 dc_no 如果为null 返回-1
采购仓库名称:	判断门店商品信息表的 dc_name 如果为null 返回其他仓
采购柜组编号:	判断门店商品信息表的 group_no 如果为null, 返回-1
采购柜组名称:	判断门店商品信息表的 group_name 如果为null, 返回其他柜组
供应商ID: 判断门店明细表的vendor_id 如果为null, 返回0
是否日清: 判断门店商品信息表的is_clear 如果为null, 返回0
写入时间: 设置为昨天
核销时间: 设置为交易日期
数据插入
-- 友好提醒: 先关联3张事实表,再去关联其他5张维度表
with t4 as(
    select
        date_format(t2.deal_date,'yyyy-MM-dd HH:mm:ss') as trade_date_time,
        date_format(t2.trade_date,'yyyy-MM-dd') as trade_date ,
        hour(t2.deal_date) as hourly,
        minute(t2.deal_date) as minute,
        t2.store_no as parent_store_no,
        coalesce(t2.sale_store_no,t2.store_no) as store_no,
        coalesce(t2.source_type,1) as source_type,
        if(
            t2.member_type like 'OL-%' OR t2.member_type like 'SF-%',
            1,
            if(
                t2.member_type = '' OR t2.member_type is null,
                0,
                2
            )
        ) as member_type,
        if(
            t3.order_no is null,
            0,
            1
        ) as is_balance_consume,
        if(
            t2.trade_id = 1,
            0,
            if(
                t2.trade_id = 2,
                2,
                5
            )
        ) as trade_type,
        if(
            t1.sale_amount < 0,
            t2.source_order_sn,
            if(
                t1.sale_amount >= 0,
                t2.parent_order_sn,
                t1.order_no
            )
        ) as parent_order_no,
       t2.order_no,
       t2.pos_no,
       t2.ser_id,
       coalesce(t1.item,0) as item,
       coalesce(t1.`sort`,0) as sort,
       date_format(t2.pay_date,'yyyy-MM-dd HH:mm:ss') as pay_time,
       date_format(t2.deal_date,'yyyy-MM-dd HH:mm:ss') as last_update_time,
       coalesce(t2.cashier_no,0)  as cashier_no,
       coalesce(t2.cashier_name,0) as cashier_name,
       coalesce(t2.member_center_sn,0) as zt_id,
       coalesce(t2.member_id,0) as member_id,
       t2.card_no,
       t1.goods_no,
        if(
            t1.combination_flag = 1,
            1,
            0
        ) as is_component,
       coalesce(t1.trade_mode_id,0) as trade_mode_id,
        t1.vendor_id,
        t1.contract_no,
        t1.is_daily_clear,
        coalesce(t1.share_user_id,0) as share_user_id,
        coalesce(t1.commission_amount,0) as commission_amount,
        t1.sale_qty,
        t1.sale_amount,
        t1.dis_amount,
        t1.sale_cost,
        if(
            t2.total_pay_amount = 0,
            0,
            if(
                t3.pay_amount is null,
                0,
                cast(t3.pay_amount * t1.sale_amount / t2.total_pay_amount as decimal(27,2))
            )
        )as balance_amount

    from (select * from ods.ods_sale_store_sale_dtl_i where combination_flag != 2 and offset_flag = 0) t1
        join ods.ods_sale_store_sale_info_i t2 on t2.order_no = t1.order_no
        left join (select * from ods.ods_sale_store_sale_pay_i where pay_type_id = '201') t3 on t2.order_no = t3.order_no
)
-- 查询并插入数据
insert overwrite table dwd.dwd_sale_store_sale_dtl_i partition(dt)
select
    t4.trade_date_time,
    t4.trade_date,
    t5.week_trade_date,
    t5.month_trade_date,
    t4.hourly,
    case
        when t4.minute between 0 and 14 then 1
        when t4.minute between 15 and 29 then 2
        when t4.minute between 30 and 44 then 3
        when t4.minute between 45 and 59 then 4
    end as quarter,
    (
        t4.hourly * 4
        +
        case
            when t4.minute between 0 and 14 then 1
            when t4.minute between 15 and 29 then 2
            when t4.minute between 30 and 44 then 3
            when t4.minute between 45 and 59 then 4
        end
    ) as quarters,
    t4.parent_store_no,
    t4.store_no,
    t7.store_name,
    t7.store_sale_type,
    t7.store_type_code,
    t7.worker_num,
    t7.store_area,
    t7.city_id,
    t7.city_name,
    t7.region_code,
    t7.region_name,
    t7.is_day_clear,
    t4.trade_type,
    t4.source_type,
    t6.source_type_name,
    case
        when t4.source_type = 1 then  1
        when t4.source_type = 9 then  5
        when t4.source_type in (4,5,6,7,8) then 6
        when t4.source_type = 11 then 8
        else 7
    end as sale_type,
    t4.member_type,
    t4.is_balance_consume,
    t4.parent_order_no,
    t4.order_no,
    t4.pos_no,
    t4.ser_id,
    t4.item,
    t4.sort,
    t4.pay_time,
    t4.last_update_time,
    t4.cashier_no,
    t4.cashier_name,
    t4.share_user_id,
    t4.commission_amount,
    t4.zt_id,
    t4.member_id,
    t4.card_no,
    t8.first_category_no,
    t8.first_category_name,
    t8.second_category_no,
    t8.second_category_name,
    t8.third_category_no,
    t8.third_category_name,
    t4.goods_no,
    t8.goods_name,
    t8.spec,
    t4.is_component,
    coalesce(t9.tag,4) as supply_team,
    coalesce(t9.dc_no,-1) as dc_no,
    coalesce(t9.dc_name,'其他仓') as dc_name,
    coalesce(t9.group_no,-1) as group_no,
    coalesce(t9.group_name,'其他柜组') as group_name ,
    t4.trade_mode_id,
    coalesce(t4.vendor_id,0) as vendor_id,
    t4.contract_no,
    coalesce(t9.is_clear,0) as is_clean,
    t4.is_daily_clear,
    t4.sale_qty,
    t4.sale_amount,
    t4.dis_amount,
    t4.sale_cost,
    t4.balance_amount,
    date_sub(current_date(),1) as write_time,
    t4.trade_date as dt
from  t4
    left join dim.dwd_dim_date_f t5 on t4.trade_date = t5.trade_date
    left join dim.dwd_dim_source_type_map_i t6 on t6.dt = '2023-11-23' and t4.source_type = t6.original_source_type
    left join dim.dwd_dim_store_i t7 on t7.dt = '2023-11-23' and t4.store_no = t7.store_no
    left join dim.dwd_dim_goods_i t8 on t8.dt = '2023-11-23' and  t4.goods_no = t8.goods_no
    left join dim.dwd_dim_store_goods_i t9 on t9.dt = '2023-11-23' and t9.goods_no = t4.goods_no;

1.1.2 线上余额支付明细

这个表的目的是为了计算dwd_sold_shop_order_dtl_i(商城核销明细表),因为是核销维度,有可能有的订单从下单到签收经历了很长时间,在判断是否是余额销售的时候,就需要取到当时下单对应的支付单的情况(因为支付单是按下单时间分区的),不可能把全量数据都拿过来做关联,所以这里就需要一个表来只记录余额支付的明细,这样就极大的降低了数据量。

处理方案: 在门店销售支付表在where条件中设置pay_channel_name = '余额支付',只保留余额支付的记录。

建表语句:
CREATE TABLE IF NOT EXISTS dwd.dwd_sale_shop_sale_balance_pay_i(
    store_no                STRING COMMENT '门店编码',
    store_name              STRING COMMENT '门店名称',
    trade_date              TIMESTAMP COMMENT '交易日期',
    member_id               BIGINT COMMENT '会员ID',
    zt_id                   BIGINT COMMENT '中台会员ID',
    trade_order_id          STRING COMMENT '关联的交易单ID',
    pay_order_id            STRING COMMENT '支付单id',
    order_no                STRING COMMENT '订单号',
    pay_channel             STRING COMMENT '支付渠道',
    pay_channel_name        STRING COMMENT '支付渠道名称',
    trade_order_type        BIGINT COMMENT '交易单类型,1消费,2充值,3提现,4退货退款',
    trade_order_type_name   STRING COMMENT '交易单类型名称',
    pay_amount              DECIMAL(27, 2) COMMENT '支付对等RMB的金额,比如是积分支付,那这里就是积分所对应的RMB的金额',
    trade_merchant          STRING COMMENT '交易商家'
)
COMMENT '线上余额支付明细'
partitioned by (dt STRING COMMENT '交易日期')
row format delimited fields terminated by ','
stored as orc
tblproperties ('orc.compress'='SNAPPY');
数据插入:
-- 线上余额支付明细
insert overwrite table dwd.dwd_sale_shop_sale_balance_pay_i partition (dt)
select
    store_no,
    store_name,
    trade_date,
    member_id,
    zt_id,
    trade_order_id,
    pay_order_id,
    order_no,
    pay_channel,
    pay_channel_name,
    trade_order_type,
    trade_order_type_name,
    pay_amount,
    trade_merchant,
    dt
from ods.ods_sale_shop_sale_pay_i where pay_channel_name = '余额支付';

-- 注意: 在后续增量的SQL中, 需要添加dt字段获取上一天的数据

1.1.3 商城订单表(核销表)

订单下单完成后,会经历各种状态变化,只要更新就会同步过来更新后的数据。因为项目有核销主题、售卖主题,在做具体的分析的时候,希望核销主题的表是按照核销时间(完成时间)进行分区的,售卖主题的表是按照售卖时间(下单时间)进行分区的。所以,这里需要对shop_order表进行处理,根据complete_time和create_time进行分别分区,用于核销主题和售卖主题使用。

建表语句:
CREATE TABLE IF NOT EXISTS dwd.dwd_sold_shop_order_i(
    id                      BIGINT COMMENT '编号',
    parent_order_no         STRING COMMENT '父单订单号',
    order_id                STRING COMMENT '订单编号',
    is_split                BIGINT COMMENT '是否拆单:0-不需要拆单;1-待拆单;2-已拆单',
    platform_id             BIGINT COMMENT '平台id:1-有赞,2-京东到家,3-美团外卖,4-饿了么',
    tid                     STRING COMMENT '平台订单号',
    source_type             BIGINT COMMENT '订单来源:10,20,30,40,41,50,60,70',
    source_name             STRING COMMENT '订单来源名称:10-有赞,20-京东到家,30-美团外卖,40-饿了么,41-百度外卖,50-传智鲜精选,60-黑马优选,70-抖店',
    store_no                STRING COMMENT '门店编码',
    city_id                 BIGINT COMMENT '城市编号',
    city_name               STRING COMMENT '城市名称',
    region_code             STRING COMMENT '区域编码',

    order_status            BIGINT,
    order_status_desc       STRING COMMENT '主订单状态描述:0-新建; 1-待出票;2-待备货;3-待揽件;4-待自提; 5-配送中;6-已完成;7-已取消',
    pay_type                BIGINT COMMENT '支付类型:1-线下支付;2-线上支付',
    trade_type              STRING COMMENT '交易类型。取值范围:fixed(一口价) gift(送礼)bulk_purchase(来自分销商的采购)present (赠品领取)group (拼团订单) pifa (批发订单) cod (货到付款) peer (代付) qrcode(扫码商家二维码直接支付的交易)qrcode_3rd(线下收银台二维码交易)',
    is_deleted              BIGINT COMMENT '是否有效,1:已删除,0:正常',

    order_create_time       TIMESTAMP COMMENT '平台订单创建时间',
    order_pay_time          TIMESTAMP COMMENT '订单支付时间',
    create_time             TIMESTAMP COMMENT '创建时间',
    print_status            BIGINT COMMENT '打印状态:0-未打票;1-已打票',
    print_time              TIMESTAMP COMMENT '出票时间',
    stock_up_status         BIGINT COMMENT '门店处理状态:0-待备货/1-已备货',
    stock_up_time           TIMESTAMP COMMENT '备货完成时间',

    order_type              BIGINT COMMENT '配送类型(真正的订单类型由业务类型来决定):1-及时送;2-隔日送;3-自提单',
    express_type            BIGINT COMMENT '配送方式:0-三方平台配送;1-自配送;2-快递;3-自提',
    receive_time            TIMESTAMP COMMENT '要求送达/自提时间',
    express_code            STRING COMMENT '配送单号',
    delivery_status         BIGINT COMMENT '配送状态:0-待配送;1-配送中;2-已送达',
    delivery_time           TIMESTAMP COMMENT '配送时间',
    pick_up_status          BIGINT COMMENT '自提状态:0-待自提;1-已自提',
    qr_code                 STRING COMMENT 'qr提货码',
    pick_up_time            TIMESTAMP COMMENT '自提时间',

    complete_time           TIMESTAMP COMMENT '订单完结时间',
    is_cancel               BIGINT COMMENT '是否取消',
    cancel_time             TIMESTAMP COMMENT '取消时间',
    cancel_reason           STRING COMMENT '取消原因',
    refund_status           BIGINT COMMENT '退款状态:0未退款,1部分退款,2已全额退款',
    refund_time             TIMESTAMP COMMENT '已退款时间',
    last_update_time        TIMESTAMP COMMENT '最新更新时间',

    order_total_amount      DECIMAL(27, 2) COMMENT '订单总金额',
    product_total_amount    DECIMAL(27, 2) COMMENT '商品总金额(原价)',
    pack_amount             DECIMAL(27, 2) COMMENT '餐盒费/打包费',
    delivery_amount         DECIMAL(27, 2) COMMENT '配送费',
    discount_amount         DECIMAL(27, 2) COMMENT '订单优惠金额=商家承担优惠金额+平台补贴金额',
    seller_discount_amount  DECIMAL(27, 2) COMMENT '商家承担优惠金额',
    platform_allowance_amount DECIMAL(27, 2) COMMENT '平台补贴金额',
    real_paid_amount        DECIMAL(27, 2) COMMENT '实付金额',
    product_discount        DECIMAL(27, 2) COMMENT '商品优惠金额',
    real_product_amount     DECIMAL(27, 2) COMMENT '商品实际金额',

    buyer_id                BIGINT COMMENT '买家id',
    buyer_phone             STRING COMMENT '买家电话',
    buyer_remark            STRING COMMENT '买家备注',
    r_name                  STRING COMMENT '收货人姓名',
    r_tel                   STRING COMMENT '收货人电话',
    r_province              STRING COMMENT '收货人省份',
    r_city                  STRING COMMENT '收货人城市',
    r_district              STRING COMMENT '收货人区域',
    r_address               STRING COMMENT '收货人地址',
    r_zipcode               STRING COMMENT '收货人邮编',

    is_tuan_head            BIGINT COMMENT '是否为团长订单',
    store_leader_id         BIGINT COMMENT '团长id',
    order_group_no          STRING COMMENT '团单号',
    commision_amount        DECIMAL(27, 2) COMMENT '抽佣金额',
    settle_amount           DECIMAL(27, 2) COMMENT '结算金额',

    points_amount           DECIMAL(27, 2) COMMENT '积分抵扣金额',
    pay_point               BIGINT COMMENT '消费积分数',
    balance_amount          DECIMAL(27, 2) COMMENT '余额扣除金额',
    pay_channel_amount      DECIMAL(27, 2) COMMENT '通过支付渠道支付的金额',
    point_amount            DECIMAL(27, 2) COMMENT '消费赠送积分',

    sync_erp_status         BIGINT COMMENT '同步erp状态',
    sync_erp_msg            STRING COMMENT '同步erp失败消息'
)
COMMENT '商城订单表(核销表)'
partitioned by (dt STRING COMMENT '完成日期')
row format delimited fields terminated by ','
stored as orc
tblproperties ('orc.compress'='SNAPPY');

ods_sale_shop_order_i 是按照最后更新时间进行分区的,所以,取出当天更新的数据,并且只取完成时间是当天的数据,插入dwd表时按照complete_time分区即可。

数据插入:
-- 商城订单表(核销表)
insert overwrite table dwd.dwd_sold_shop_order_i partition (dt)
select
    id,
    parent_order_no,
    order_id,
    is_split,
    platform_id,
    tid,
    source_type,
    source_name,
    store_no,
    city_id,
    city_name,
    region_code,
    order_status,
    order_status_desc,
    pay_type,
    trade_type,
    is_deleted,
    order_create_time,
    order_pay_time,
    create_time,
    print_status,
    print_time,
    stock_up_status,
    stock_up_time,
    order_type,
    express_type,
    receive_time,
    express_code,
    delivery_status,
    delivery_time,
    pick_up_status,
    qr_code,
    pick_up_time,
    complete_time,
    is_cancel,
    cancel_time,
    cancel_reason,
    refund_status,
    refund_time,
    last_update_time,
    order_total_amount,
    product_total_amount,
    pack_amount,
    delivery_amount,
    discount_amount,
    seller_discount_amount,
    platform_allowance_amount,
    real_paid_amount,
    product_discount,
    real_product_amount,
    buyer_id,
    buyer_phone,
    buyer_remark,
    r_name,
    r_tel,
    r_province,
    r_city,
    r_district,
    r_address,
    r_zipcode,
    is_tuan_head,
    store_leader_id,
    order_group_no,
    commision_amount,
    settle_amount,
    points_amount,
    pay_point,
    balance_amount,
    pay_channel_amount,
    point_amount,
    sync_erp_status,
    sync_erp_msg,
    date_format(complete_time,'yyyy-MM-dd') as dt

from ods.ods_sale_shop_order_i where complete_time is not null;  

-- 后续增量只需加上where dt = '2023-11-24' and  date_format(complete_time,'yyyy-MM-dd') = '2023-11-24';

1.1.4 订单明细表(核销表)

与商城订单表一样, 只获取核销后的订单数据, 以及是最终完结的数据, 在售卖主题中, 获取下单时的信息数据即可

建表语句:
CREATE TABLE IF NOT EXISTS dwd.dwd_sold_shop_order_item_i(
    id                          BIGINT COMMENT '自增id',
    order_id                    STRING COMMENT '订单编号',
    goods_no                    STRING COMMENT '商品编码',
    goods_name                  STRING COMMENT '商品名称',

    weight                      DECIMAL(27, 3) COMMENT '重量,单位kg',
    quantity                    BIGINT COMMENT '数量',
    unit                        STRING COMMENT '单位',
    sale_qty                    DECIMAL(27, 3) COMMENT '销售数量',
    disp_price                  DECIMAL(27, 2) COMMENT 'sku展示价格',
    pay_price                   DECIMAL(27, 2) COMMENT '价格',
    sale_amount                 DECIMAL(27, 2) COMMENT '单品销售金额',
    dis_amount                  DECIMAL(27, 2) COMMENT '单品总折扣金额',
    sale_cost                   DECIMAL(27, 2) COMMENT '销售成本',

    sale_type                   BIGINT COMMENT '类型:1-常规;2-赠品',
    create_time                 TIMESTAMP COMMENT '创建时间',
    complete_time               TIMESTAMP COMMENT '完成时间',
    last_update_time            TIMESTAMP COMMENT '更新时间',

    activity_plat_city_goods_id BIGINT COMMENT '活动商品区域id',
    activity_type               BIGINT COMMENT '活动类型(11:拼团 21:秒杀)',
    item_goods_key              STRING COMMENT '虚拟字段,itemgoodskey',
    is_deleted                  BIGINT COMMENT '是否删除:0-否;1-删除',

    transfer_paper_no           STRING COMMENT '要货单号',
    serial_no                   BIGINT COMMENT '商品序号,每个订单下起始都为1',
    is_delivery                 BIGINT COMMENT '仓发是否配送;1:配送',
    goods_source_type           BIGINT COMMENT '商品来源类型:1-生鲜品;2-标品',

    trade_mode_id               BIGINT COMMENT '结算方式:1购销,2联营',
    vendor_id                   BIGINT COMMENT '供应商ID',
    contract_no                 STRING COMMENT '合同编号'
)
COMMENT '订单明细表(核销表)'
partitioned by (dt STRING COMMENT '完成日期')
row format delimited fields terminated by ','
stored as orc
tblproperties ('orc.compress'='SNAPPY');

处理方式与商城订单表也是类似的

数据插入:
-- 商城订单明细表
insert overwrite table dwd.dwd_sold_shop_order_item_i partition (dt)
select
    id,
    order_id,
    goods_no,
    goods_name,
    weight,
    quantity,
    unit,
    sale_qty,
    disp_price,
    pay_price,
    sale_amount,
    dis_amount,
    sale_cost,
    sale_type,
    create_time,
    complete_time,
    last_update_time,
    activity_plat_city_goods_id,
    activity_type,
    item_goods_key,
    is_deleted,
    transfer_paper_no,
    serial_no,
    is_delivery,
    goods_source_type,
    trade_mode_id,
    vendor_id,
    contract_no,
    date_format(complete_time,'yyyy-MM-dd') as dt
from ods.ods_sale_shop_order_item_i where complete_time is not null;  

-- 后续增量只需加上where dt = '2023-11-24' and  date_format(complete_time,'yyyy-MM-dd') = '2023-11-24';

1.1.5 商城核销明细宽表

建表语句:
CREATE TABLE IF NOT EXISTS dwd.dwd_sold_shop_order_dtl_i(
    complete_time           STRING COMMENT '订单完成时间',
    trade_date              STRING COMMENT '交易日期',
    -- 日期表
    week_trade_date         STRING COMMENT '周一日期',
    month_trade_date        STRING COMMENT '月一日期',
    
    hourly                  BIGINT COMMENT '交易小时(0-23)',
    quarter                 BIGINT COMMENT '刻钟:1.0-15,2.15-30,3.30-45,4.45-60',
    quarters                BIGINT COMMENT '刻钟数:hourly*4+quarters',
    
    parent_order_no         STRING COMMENT '父单订单号/源单号',
    order_id                STRING COMMENT '订单编号',
    trade_type              BIGINT COMMENT '结算类型(0.正常交易,1.赠品发放,2.退货,4.培训,5.取消交易)',
    is_split                BIGINT COMMENT '是否拆单:0-不需要拆单;1-待拆单;2-已拆单',
    platform_id             BIGINT COMMENT '平台id:1-有赞,2-京东到家,3-美团外卖,4-饿了么',
    tid                     STRING COMMENT '平台订单号',
    source_type             BIGINT COMMENT '订单来源:10,20,30,40,41,50,60,70',
    source_name             STRING COMMENT '订单来源名称:10-有赞,20-京东到家,30-美团外卖,40-饿了么,41-百度外卖,50-传智鲜精选,60-黑马优选,70-抖店',
    order_type              BIGINT COMMENT '配送类型(真正的订单类型由业务类型来决定):1-及时送;2-隔日送;3-自提单',
    express_type            BIGINT COMMENT '配送方式:0-三方平台配送;1-自配送;2-快递;3-自提',
    
    order_status            BIGINT,
    order_status_desc       STRING COMMENT '主订单状态描述:0-新建; 1-待出票;2-待备货;3-待揽件;4-待自提; 5-配送中;6-已完成;7-已取消',
    pay_type                BIGINT COMMENT '支付类型:1-线下支付;2-线上支付',
    is_balance_consume      BIGINT COMMENT '是否余额支付:1是,0否',
    
    store_no                STRING COMMENT '店铺编码',
    -- 分店信息表
    store_name              STRING COMMENT '店铺名称',
    store_sale_type         BIGINT COMMENT '店铺销售类型',
    store_type_code         BIGINT COMMENT '分店类型',
    worker_num              BIGINT COMMENT '员工人数',
    store_area              DECIMAL(27, 2) COMMENT '门店面积',
    city_id                 BIGINT COMMENT '城市ID',
    city_name               STRING COMMENT '城市名称',
    region_code             STRING COMMENT '区域编码',
    region_name             STRING COMMENT '区域名称',
    is_day_clear            BIGINT COMMENT '是否日清:0否,1是',
    
    order_create_time       TIMESTAMP COMMENT '平台订单创建时间',
    order_pay_time          TIMESTAMP COMMENT '订单支付时间',
    create_time             TIMESTAMP COMMENT '创建时间',
    is_cancel               BIGINT COMMENT '是否取消',
    cancel_time             TIMESTAMP COMMENT '取消时间',
    cancel_reason           STRING COMMENT '取消原因',
    last_update_time        TIMESTAMP COMMENT '最新更新时间',
    -- 会员基础信息表
    zt_id                   BIGINT COMMENT '中台ID',
    
    buyer_id                BIGINT COMMENT '买家id',
    buyer_phone             STRING COMMENT '买家电话',
    buyer_remark            STRING COMMENT '买家备注',
    r_name                  STRING COMMENT '收货人姓名',
    r_tel                   STRING COMMENT '收货人电话',
    r_province              STRING COMMENT '收货人省份',
    r_city                  STRING COMMENT '收货人城市',
    r_district              STRING COMMENT '收货人区域',
    r_address               STRING COMMENT '收货人地址',
    r_zipcode               STRING COMMENT '收货人邮编',
    
    is_tuan_head            BIGINT COMMENT '是否为团长订单',
    store_leader_id         BIGINT COMMENT '团长id',
    order_group_no          STRING COMMENT '团单号',
    commission_amount       DECIMAL(27, 2) COMMENT '抽佣金额',
    settle_amount           DECIMAL(27, 2) COMMENT '结算金额',
   	-- 商品表 
    first_category_no       STRING COMMENT '一级分类编码',
    first_category_name     STRING COMMENT '一级分类名称',
    second_category_no      STRING COMMENT '二级分类编码',
    second_category_name    STRING COMMENT '二级分类名称',
    third_category_no       STRING COMMENT '三级分类编码',
    third_category_name     STRING COMMENT '三级分类名称',
    goods_no                STRING COMMENT '商品编码',
    goods_name              STRING COMMENT '商品名称',
    
    weight                  DECIMAL(27, 3) COMMENT '重量,单位kg',
    quantity                DECIMAL(27, 3) COMMENT '数量',
    unit                    STRING COMMENT '单位',
    sale_qty                DECIMAL(27, 3) COMMENT '销售数量',
    disp_price              DECIMAL(27, 2) COMMENT 'sku展示价格',
    pay_price               DECIMAL(27, 2) COMMENT '价格',
    sale_amount             DECIMAL(27, 2) COMMENT '单品销售金额',
    dis_amount              DECIMAL(27, 2) COMMENT '单品总折扣金额',
    sale_cost               DECIMAL(27, 2) COMMENT '销售成本',
    
    sale_type               BIGINT COMMENT '类型:1-常规;2-赠品',
    activity_plat_city_goods_id BIGINT COMMENT '活动商品区域id',
    activity_type           BIGINT COMMENT '活动类型(11:拼团 21:秒杀)',
    
    order_total_amount      DECIMAL(27, 2) COMMENT '订单总金额(平摊)',
    order_discount_amount   DECIMAL(27, 2) COMMENT '订单优惠金额=商家承担优惠金额+平台补贴金额(平摊)',
    order_paid_amount       DECIMAL(27, 2) COMMENT '实付金额(平摊)',
    balance_amount          DECIMAL(27, 2) COMMENT '余额支付',
    -- 门店商品信息表
    supply_team             BIGINT COMMENT '供应链团队 1.平台商品,2.优选标品,3.传智鲜标品,4.传智鲜生鲜,5优选POS商品',
    dc_no                   STRING COMMENT '采购仓库编号',
    dc_name                 STRING COMMENT '采购仓库名称',
    group_no                STRING COMMENT '采购柜组编号',
    group_name              STRING COMMENT '采购柜组名称',
    
    trade_mode_id           BIGINT COMMENT '结算方式:1购销,2联营',
    vendor_id               BIGINT COMMENT '供应商ID',
    contract_no             STRING COMMENT '合同编号'
)
COMMENT '商城核销明细表'
partitioned by (dt STRING COMMENT '核销日期')
row format delimited fields terminated by ','
stored as orc
tblproperties ('orc.compress'='SNAPPY');
需求梳理:

【黑马甄选离线数仓day06_核销主题域开发】_第3张图片

一共需要5张事实表:
	dwd_sold_shop_order_i、
	dwd_sold_shop_order_item_i、
	dwd_sale_shop_sale_balance_pay_i: dwd_sale_shop_sale_balance_pay_i用order_no与订单表和订单明细表的order_id关联。
	ods_sale_shop_refund_i 退款表,主要记录退款单号、时间、原因、金额等信息。
	ods_sale_shop_refund_item_i 退款明细表,主要记录退款的商品信息以及活动信息等。

关联事实表思路:

先 完成正常订单的 关联 , 再完成 退款订单的关联 ,最后两个结果合并到一起是5张事实表的最终关联结果

1- 先将 商城订单表、订单明细表、线上余额支付明细 3个表进行关联: 其处理操作与门店销售明细宽表类似 1- 当is_cancel 为1是, 返回5 否则返回0 2- 是否为余额支付, 判断支付表即可 3- commission_amount、settle_amount order_total_amount order_discount_amount order_paid_amount balance_amount 需要计算

2- 接着将 退款表、退款明细表、商城订单表以及线上余额支付明细表 4个表 进行进行关联: 其处理操作与门店销售明细宽表类似 1- 日期使用退款的创建时间即可 2- trade_type(结算类型), 如果cancel_time不为null, 返回5, 否则返回2 3- 需要将计算后的commission_amount(抽佣金额), settle_amount(结算金额)调整为负数 5- quantity,qty,amount,cost调整为负数(做对冲(冲减单)) 6- weight,unit,dis_amount,order_discount_amount调整为0 7- sale_type类型只有1 8- unit设置为空即可

3- 将1和2的结果 基于union all 关联在一起, 然后和五张维度表进行关联: 其处理操作与门店销售明细宽表类似

1.先关联 正常订单 涉及的表:

	商城订单表: dwd_sold_shop_order_i、
	商城订单明细表: dwd_sold_shop_order_item_i、
	线上余额支付明细表: dwd_sale_shop_sale_balance_pay_i
	
	3表关联条件: dwd_sale_shop_sale_balance_pay_i用order_no与订单表和订单明细表的order_id关

【黑马甄选离线数仓day06_核销主题域开发】_第4张图片

2.再关联 退款订单 涉及的表:

因为退款单要获取原单的记录,所以退款表还要与订单表进行关联来取一些字段。比较庆幸的一点是,在退款的时候,订单表也会进行更新,这样在关联订单的时候,只要在对应的那个分区里就可以找到对应的记录,不用再去遍历多个分区来找。

	商城退款表:ods_sale_shop_refund_i 
	商城退款明细表:ods_sale_shop_refund_item_i 
	商城订单表: dwd_sold_shop_order_i
	线上余额支付明细表: dwd_sale_shop_sale_balance_pay_i

【黑马甄选离线数仓day06_核销主题域开发】_第5张图片

  1. 将1和2的结果 基于union all 关联在一起, 然后和五张维度表进行关联: 其处理操作与门店销售明细宽表类似

需要五张维表:
	dwd_dim_date_f
	dwd_dim_store_i
	dwd_dim_goods_i
	dwd_dim_store_goods_i
	ods_mem_member_union_i  注意:会员基础信息表,记录了会员的ID,联系方式,归属门店,来源,状态等信息。

数据插入:
-- 商城核销明细宽表:
with t5 as (
    select
        t1.complete_time,
        date_format(t1.complete_time,'yyyy-MM-dd') as trade_date,

        hour(t1.complete_time) as hourly,
        minute(t1.complete_time) as minute,

        t1.parent_order_no,
        t1.order_id,
        t1.trade_type,
        t1.is_split,
        t1.platform_id,
        t1.tid,
        t1.source_type,
        t1.source_name,
        t1.order_type,
        t1.express_type,
        t1.order_status,
        t1.order_status_desc,
        t1.pay_type,
        if(
            t3.order_no is not null,
            1,
            0
        ) as is_balance_consume,
        t1.store_no,

        t1.order_create_time,
        t1.order_pay_time,
        t1.create_time,
        if(
            t1.is_cancel = 1,
            5,
            0
        ) as is_cancel,
        t1.cancel_time,
        t1.cancel_reason,
        t1.last_update_time,
        t1.buyer_id,
        t1.buyer_phone,
        t1.buyer_remark,
        t1.r_name,
        t1.r_tel,
        t1.r_province,
        t1.r_city,
        t1.r_district,
        t1.r_address,
        t1.r_zipcode,
        t1.is_tuan_head,
        t1.store_leader_id,
        t1.order_group_no,
        t1.commision_amount * t2.sale_amount / t1.real_product_amount as commission_amount, -- 抽佣金额
        t1.settle_amount * t2.sale_amount / t1.real_product_amount  as settle_amount,  -- 结算金额
        t2.goods_no,
        t2.weight,
        t2.quantity,
        t2.unit,
        t2.sale_qty,
        t2.disp_price,
        t2.pay_price,
        t2.sale_amount,
        t2.dis_amount,
        t2.sale_cost,
        t2.sale_type,
        t2.activity_plat_city_goods_id,
        t2.activity_type,
        t1.order_total_amount *  t2.sale_amount / t1.real_product_amount as order_total_amount, -- 订单总金额(平摊)
        t1.discount_amount * t2.sale_amount / t1.real_product_amount as order_discount_amount, -- 订单优惠金额 = 商家承担优惠金额 + 平台补贴金额 (平摊)
        if(
            t1.real_paid_amount = 0,
            0,
            t1.real_paid_amount * t2.sale_amount / t1.real_product_amount
        ) as order_paid_amount, -- 实付金额(平摊)
        if(
            t1.real_paid_amount = 0,
            0,
            if(
                t3.pay_amount is null,
                0,
                t3.pay_amount * t2.sale_amount / t1.real_product_amount
            )

        ) as balance_amount,  -- 余额支付金额 (平摊) 首先判断单据金额是否等于0 , 则结果为0, 如果不是, 判断是否有余额支付金额, 如果有, 使用余额金额 * 商品销售金额 / 单据金额

        t2.trade_mode_id,
        t2.vendor_id,
        t2.contract_no,
        t1.dt
    from dwd.dwd_sold_shop_order_i t1
        join dwd.dwd_sold_shop_order_item_i t2 on t1.order_id = t2.order_id
        left join dwd.dwd_sale_shop_sale_balance_pay_i t3 on t1.order_id = t3.order_no
    
    -- 合并
    union all
    
    -- 步骤二:
    select
        t1.create_time as complete_time,
        date_format(t1.create_time,'yyyy-MM-dd') as trade_date,

        hour(t1.create_time) as hourly,
        minute(t1.create_time) as minute,

        t3.parent_order_no,
        t3.order_id,
        if(
            t1.cancel_time is not null,
            '5',
            '2'
        ) as trade_type,
        t3.is_split,
        t3.platform_id,
        t3.tid,
        t3.source_type,
        t3.source_name,
        t3.order_type,
        t3.express_type,
        t3.order_status,
        t3.order_status_desc,
        t3.pay_type,
        if(
            t1.order_no is not null,
            1,
            0
        ) as is_balance_consume,
        t1.store_no,

        t3.order_create_time,
        t3.order_pay_time,
        t3.create_time,
        if(
            t3.is_cancel = 1,
            5,
            0
        ) as is_cancel,
        t1.cancel_time,
        t3.cancel_reason,
        t3.last_update_time,
        t3.buyer_id,
        t3.buyer_phone,
        t3.buyer_remark,
        t3.r_name,
        t3.r_tel,
        t3.r_province,
        t3.r_city,
        t3.r_district,
        t3.r_address,
        t3.r_zipcode,
        t3.is_tuan_head,
        t1.store_leader_id,
        t3.order_group_no,
        -(t3.commision_amount * t2.amount / t3.real_product_amount) as commission_amount, -- 抽佣金额
        -(t3.settle_amount * t2.amount / t3.real_product_amount)  as settle_amount,  -- 结算金额
        t2.goods_no,
        0 as weight,
        -t2.quantity,
        '' as unit,
        -t2.qty,
        t2.amount / t2.quantity as disp_price,
        t2.amount / t2.quantity as pay_price,
        -t2.amount ,
        0 as dis_amount,
        -t2.cost as sale_cost,
        1 as sale_type,
        t2.activity_plat_city_goods_id,
        t2.activity_type,
        -t2.amount as order_total_amount, -- 订单总金额(平摊)
        0 as order_discount_amount, -- 订单优惠金额 = 商家承担优惠金额 + 平台补贴金额 (平摊)
        -t2.amount as order_paid_amount, -- 实付金额(平摊)
        - if(
            t3.real_paid_amount = 0,
            0,
            if(
                t4.pay_amount is null,
                0,
                t4.pay_amount * t2.amount / t3.real_product_amount
            )

        ) as balance_amount,  -- 余额支付金额 (平摊) 首先判断单据金额是否等于0 , 则结果为0, 如果不是, 判断是否有余额支付金额, 如果有, 使用余额金额 * 商品销售金额 / 单据金额

        t2.trade_mode_id,
        t2.vendor_id,
        t2.contract_no,
        t1.dt
    from ods.ods_sale_shop_refund_i t1
        join ods.ods_sale_shop_refund_item_i t2 on t1.refund_no = t2.refund_no
        left join dwd.dwd_sold_shop_order_i t3 on t1.order_no = t3.order_id
        left join dwd.dwd_sale_shop_sale_balance_pay_i t4 on t4.order_no = t1.order_no
)

insert overwrite table dwd.dwd_sold_shop_order_dtl_i partition (dt)
select
    t5.complete_time,
    t5.trade_date,
    t6.week_trade_date,
    t6.month_trade_date,
    t5.hourly,
    case
        when t5.minute between 0 and 14 then 1
        when t5.minute between 15 and 29 then 2
        when t5.minute between 30 and 44 then 3
        when t5.minute between 45 and 59 then 4
    end as quarter,
    (
        t5.hourly * 4
        +
        case
            when t5.minute between 0 and 14 then 1
            when t5.minute between 15 and 29 then 2
            when t5.minute between 30 and 44 then 3
            when t5.minute between 45 and 59 then 4
        end
    ) as quarters,
    t5.parent_order_no,
    t5.order_id,
    t5.trade_type,
    t5.is_split,
    t5.platform_id,
    t5.tid,
    t5.source_type,
    t5.source_name,
    t5.order_type,
    t5.express_type,
    t5.order_status,
    t5.order_status_desc,
    t5.pay_type,
    t5.is_balance_consume,
    t5.store_no,
    t7.store_name,
    t7.store_sale_type,
    t7.store_type_code,
    t7.worker_num,
    t7.store_area,
    t7.city_id,
    t7.city_name,
    t7.region_code,
    t7.region_name,
    t7.is_day_clear,
    t5.order_create_time,
    t5.order_pay_time,
    t5.create_time,
    t5.is_cancel,
    t5.cancel_time,
    t5.cancel_reason,
    t5.last_update_time,
    t10.zt_id,
    t5.buyer_id,
    t5.buyer_phone,
    t5.buyer_remark,
    t5.r_name,
    t5.r_tel,
    t5.r_province,
    t5.r_city,
    t5.r_district,
    t5.r_address,
    t5.r_zipcode,
    t5.is_tuan_head,
    t5.store_leader_id,
    t5.order_group_no,
    t5.commission_amount,
    t5.settle_amount,
    t8.first_category_no,
    t8.first_category_name,
    t8.second_category_no,
    t8.second_category_name,
    t8.third_category_no,
    t8.third_category_name,
    t5.goods_no,
    t8.goods_name,
    t5.weight,
    t5.quantity,
    t5.unit,
    t5.sale_qty,
    t5.disp_price,
    t5.pay_price,
    t5.sale_amount,
    t5.dis_amount,
    t5.sale_cost,
    t5.sale_type,
    t5.activity_plat_city_goods_id,
    t5.activity_type,
    t5.order_total_amount,
    t5.order_discount_amount,
    t5.order_paid_amount,
    t5.balance_amount,
    coalesce(t9.tag,4) as supply_team,
    coalesce(t9.dc_no,-1) as dc_no,
    coalesce(t9.dc_name,'其他仓') as dc_name,
    coalesce(t9.group_no,-1) as group_no,
    coalesce(t9.group_name,'其他柜组') as group_name ,
    t5.trade_mode_id,
    coalesce(t5.vendor_id,0) as vendor_id,
    t5.contract_no,
    t5.dt
from t5
    left join dim.dwd_dim_date_f t6 on t5.trade_date = t6.trade_date
    left join dim.dwd_dim_store_i t7 on t5.store_no = t7.store_no and t7.dt = '2023-11-23'
    left join dim.dwd_dim_goods_i t8 on t5.goods_no = t8.goods_no and t8.dt = '2023-11-23'
    left join dim.dwd_dim_store_goods_i t9 on t8.goods_no = t9.goods_no and t7.store_no = t9.store_no and t9.dt = '2023-11-23'
    left join ods.ods_mem_member_union_i t10 on t5.buyer_id = t10.member_id;

1.1.6 门店库调表

库调即库存调整,类型有日清活动、盘点更正、报损/溢等。对于这些情况,都是广义上的损耗。为了计算损耗,需要对库调表进行加宽处理。

需求说明: 对源表进行加宽处理,处理时间、添加门店信息等。

建表语句:
CREATE TABLE IF NOT EXISTS dwd.dwd_stock_store_stock_adj_i(
    trade_date              STRING COMMENT '库存处理日期',
    week_trade_date         STRING COMMENT '周一日期',
    month_trade_date        STRING COMMENT '月一日期',
    hourly                  BIGINT COMMENT '交易小时(0-23)',
    quarter                 BIGINT COMMENT '刻钟:1.0-15,2.15-30,3.30-45,4.45-60',
    quarters                BIGINT COMMENT '刻钟数:hourly*4+quarters',

    id                      BIGINT COMMENT '主键',
    uid                     STRING COMMENT '唯一标识',
    order_id                STRING COMMENT '库调单号',
    order_source            BIGINT COMMENT '下单来源,小程序、pc等', 

    store_no                STRING COMMENT '店铺编码',
    store_name              STRING COMMENT '店铺名称',
    store_sale_type         BIGINT COMMENT '店铺销售类型',
    store_type_code         BIGINT COMMENT '分店类型',
    worker_num              BIGINT COMMENT '员工人数',
    store_area              DECIMAL(27, 2) COMMENT '门店面积',
    city_id                 BIGINT COMMENT '城市ID',
    city_name               STRING COMMENT '城市名称',
    region_code             STRING COMMENT '区域编码',
    region_name             STRING COMMENT '区域名称',
    is_day_clear            BIGINT COMMENT '是否日清:0否,1是',

    goods_no                STRING COMMENT '商品编码',
    goods_name              STRING COMMENT '商品名称',
    adj_type_big            STRING COMMENT '库调类型(大类)',
    adj_type_small          STRING COMMENT '库调类型(小类)',
    adj_reason_big          STRING COMMENT '库调原因(大类)',
    adj_reason_small        STRING COMMENT '库调原因(小类)',
    adj_qty                 DECIMAL(27, 3) COMMENT '库调数量',
    adj_price               DECIMAL(27, 2) COMMENT '库调单价',
    adj_amount              DECIMAL(27, 2) COMMENT '库调金额',
    create_time             TIMESTAMP COMMENT '创建时间',
    stock_deal_time         TIMESTAMP COMMENT '库存处理时间',
    sync_time               TIMESTAMP COMMENT '数据同步时间',
    vendor_no               STRING COMMENT '供应商编码',
    vendor_name             STRING COMMENT '供应商名称'
) 
COMMENT '门店库调单'
partitioned by(dt STRING COMMENT '库存处理时间')
row format delimited fields terminated by ','
stored as orc
tblproperties ('orc.compress'='SNAPPY');

涉及表: 门店库调表 和 日期表以及分店信息表 进行关联

分析处理:

时间采用库存处理时间
数据插入:
insert overwrite table dwd.dwd_stock_store_stock_adj_i partition (dt)
select
    t2.trade_date,
    t2.week_trade_date,
    t2.month_trade_date,
    hour(t1.stock_deal_time) as hourly,
    case
        when minute(t1.stock_deal_time) between 0 and 14 then 1
        when minute(t1.stock_deal_time) between 15 and 29 then 2
        when minute(t1.stock_deal_time) between 30 and 44 then 3
        when minute(t1.stock_deal_time) between 45 and 59 then 4
    end as quarter,
    (
        hour(t1.stock_deal_time) * 4
        +
        case
            when minute(t1.stock_deal_time) between 0 and 14 then 1
            when minute(t1.stock_deal_time) between 15 and 29 then 2
            when minute(t1.stock_deal_time) between 30 and 44 then 3
            when minute(t1.stock_deal_time) between 45 and 59 then 4
        end
    ) as quarters,
    t1.id,
    t1.uid,
    t1.order_id,
    t1.order_source,
    t1.store_no,
    t1.store_name,
    t3.store_sale_type,
    t3.store_type_code,
    t3.worker_num,
    t3.store_area,
    t3.city_id,
    t3.city_name,
    t3.region_code,
    t3.region_name,
    t3.is_day_clear,
    t1.goods_no,
    t1.goods_name,
    t1.adj_type_big,
    t1.adj_type_small,
    t1.adj_reason_big,
    t1.adj_reason_small,
    t1.adj_qty,
    t1.adj_price,
    t1.adj_amount,
    t1.create_time,
    t1.stock_deal_time,
    t1.sync_time,
    t1.vendor_no,
    t1.vendor_name,
    t2.trade_date as dt
from ods.ods_stock_store_stock_adj_i t1
    left join dim.dwd_dim_date_f t2 on  date_format(t1.stock_deal_time,'yyyy-MM-dd') = t2.trade_date
    left join dim.dwd_dim_store_i t3 on t3.dt = '2023-11-23' and t1.store_no = t3.store_no;

1.1.7 门店收货单

门店的收货情况,包括单号,时间,商品信息,数量和金额等。

涉及表: ods_order_store_receive_i(门店收货单)、时间表、分店信息表关联

建表语句:
CREATE TABLE IF NOT EXISTS dwd.dwd_order_store_receive_i(
    trade_date              STRING COMMENT '库存处理日期',
    week_trade_date         STRING COMMENT '周一日期',
    month_trade_date        STRING COMMENT '月一日期',
    hourly                  BIGINT COMMENT '交易小时(0-23)',
    quarter                 BIGINT COMMENT '刻钟:1.0-15,2.15-30,3.30-45,4.45-60',
    quarters                BIGINT COMMENT '刻钟数:hourly*4+quarters',

    id                      BIGINT COMMENT '主键',
    uid                     STRING COMMENT '唯一标识',
    order_id                STRING COMMENT '收货单号',
    order_source            BIGINT COMMENT '下单来源,小程序、pc等',

    store_no                STRING COMMENT '店铺编码',
    store_name              STRING COMMENT '店铺名称',
    store_sale_type         BIGINT COMMENT '店铺销售类型',
    store_type_code         BIGINT COMMENT '分店类型',
    worker_num              BIGINT COMMENT '员工人数',
    store_area              DECIMAL(27, 2) COMMENT '门店面积',
    city_id                 BIGINT COMMENT '城市ID',
    city_name               STRING COMMENT '城市名称',
    region_code             STRING COMMENT '区域编码',
    region_name             STRING COMMENT '区域名称',
    is_day_clear            BIGINT COMMENT '是否日清:0否,1是',

    goods_no                STRING COMMENT '商品编码',
    goods_name              STRING COMMENT '商品名称',
    dc_no                   STRING COMMENT '配送中心编码',
    dc_name                 STRING COMMENT '配送中心名称',
    vendor_no               STRING COMMENT '供应商编码',
    vendor_name             STRING COMMENT '供应商名称',
    order_type              BIGINT COMMENT '订单类型,1-直送,2-配送,3-代发',
    receive_price           DECIMAL(27, 2) COMMENT '收货价',
    receive_qty             DECIMAL(27, 3) COMMENT '收货数量',
    git_qty                 DECIMAL(27, 3) COMMENT '赠品数量',
    create_time             TIMESTAMP COMMENT '创建时间',
    stock_deal_time         TIMESTAMP COMMENT '库存处理时间',
    dc_send_order_id        STRING COMMENT '仓库发货单号',
    red_order_id            STRING COMMENT '被红冲单号',
    contract_no             STRING COMMENT '合同编号',
    contract_name           STRING COMMENT '合同名称',
    trade_mode              BIGINT COMMENT '1-直营,2-联营',
    order_source_type       BIGINT COMMENT '订货标识,0-门店订货,1-采购配货',
    sync_time               TIMESTAMP COMMENT '数据同步时间'
)
COMMENT '门店收货单'
partitioned by (dt STRING COMMENT '库存处理时间')
row format delimited fields terminated by ','
stored as orc
tblproperties ('orc.compress'='SNAPPY');
数据插入:

计算处理: 日期字段 stock_deal_time

insert overwrite table dwd.dwd_order_store_receive_i partition(dt)
select
    date_format(t.stock_deal_time,'yyyy-MM-dd') as trade_date
     ,dd.week_trade_date
     ,dd.month_trade_date
     ,hour(t.stock_deal_time) as hourly
     ,case when minute(t.stock_deal_time)>=0  and minute(t.stock_deal_time)<15 then 1
           when minute(t.stock_deal_time)>=15 and minute(t.stock_deal_time)<30 then 2
           when minute(t.stock_deal_time)>=30 and minute(t.stock_deal_time)<45 then 3
           when minute(t.stock_deal_time)>=45 and minute(t.stock_deal_time)<60 then 4
    end as `quarter`
     ,hour(t.stock_deal_time)*4 + case when minute(t.stock_deal_time)>=0  and minute(t.stock_deal_time)<15 then 1
                                       when minute(t.stock_deal_time)>=15 and minute(t.stock_deal_time)<30 then 2
                                       when minute(t.stock_deal_time)>=30 and minute(t.stock_deal_time)<45 then 3
                                       when minute(t.stock_deal_time)>=45 and minute(t.stock_deal_time)<60 then 4
    end  as quarters

     ,t.id
     ,t.uid
     ,t.order_id
     ,t.order_source

     ,t.store_no
     ,bs.store_name
     ,bs.store_sale_type
     ,bs.store_type_code
     ,bs.worker_num
     ,bs.store_area
     ,bs.city_id
     ,bs.city_name
     ,bs.region_code
     ,bs.region_name
     ,bs.is_day_clear

     ,t.goods_no
     ,t.goods_name
     ,t.dc_no
     ,t.dc_name
     ,t.vendor_no
     ,t.vendor_name
     ,t.order_type
     ,t.receive_price
     ,t.receive_qty
     ,t.git_qty
     ,t.create_time
     ,t.stock_deal_time
     ,t.dc_send_order_id
     ,t.red_order_id
     ,t.contract_no
     ,t.contract_name
     ,t.trade_mode
     ,t.order_source_type
     ,t.sync_time
     ,date_format(t.stock_deal_time,'yyyy-MM-dd') as dt
from ods.ods_order_store_receive_i t
inner join dim.dwd_dim_date_f as dd
 on date_format(t.stock_deal_time,'yyyy-MM-dd')=dd.trade_date
inner join dim.dwd_dim_store_i as bs
 on t.store_no=bs.store_no and bs.dt= '2023-11-23';

1.1.8 门店退货单

门店退给供应商的商品情况,包括单号、门店、时间、商品信息、数量、金额、原因等。

涉及表: ods_order_store_return_to_vendor_i(门店退货单)、时间表、门店信息表

建表语句:
CREATE TABLE IF NOT EXISTS dwd.dwd_order_store_return_to_vendor_i(
    trade_date              STRING COMMENT '库存处理日期',
    week_trade_date         STRING COMMENT '周一日期',
    month_trade_date        STRING COMMENT '月一日期',
    hourly                  BIGINT COMMENT '交易小时(0-23)',
    quarter                 BIGINT COMMENT '刻钟:1.0-15,2.15-30,3.30-45,4.45-60',
    quarters                BIGINT COMMENT '刻钟数:hourly*4+quarters',

    id                      BIGINT COMMENT '主键',
    uid                     STRING COMMENT '唯一标识',
    order_id                STRING COMMENT '退配单号',
    order_source            BIGINT COMMENT '下单来源,小程序、pc、智能补货、系统等',

    store_no                STRING COMMENT '门店编码',
    store_name              STRING COMMENT '门店名称',
    store_sale_type         BIGINT COMMENT '店铺销售类型',
    store_type_code         BIGINT COMMENT '分店类型',
    worker_num              BIGINT COMMENT '员工人数',
    store_area              DECIMAL(27, 2) COMMENT '门店面积',
    city_id                 BIGINT COMMENT '城市ID',
    city_name               STRING COMMENT '城市名称',
    region_code             STRING COMMENT '区域编码',
    region_name             STRING COMMENT '区域名称',
    is_day_clear            BIGINT COMMENT '是否日清:0否,1是',

    goods_no                STRING COMMENT '商品编码',
    goods_name              STRING COMMENT '商品名称',
    dc_no                   STRING COMMENT '配送中心编码',
    dc_name                 STRING COMMENT '配送中心名称',
    vendor_no               STRING COMMENT '供应商编码',
    vendor_name             STRING COMMENT '供应商名称',
    return_price            DECIMAL(27, 2) COMMENT '退配价',
    return_qty              DECIMAL(27, 3) COMMENT '退配数量',
    create_time             TIMESTAMP COMMENT '创建时间',
    stock_deal_time         TIMESTAMP COMMENT '库存处理时间',
    original_order_id       STRING COMMENT '退配原单号',
    is_fresh                BIGINT COMMENT '是否为生鲜店,0-否,1-是',
    is_entity               BIGINT COMMENT '是否实物退回,0-否,1-是',
    responsible_person      STRING COMMENT '责任归属方',
    return_reason_big       STRING COMMENT '退配原因(大类)',
    return_desc_big         STRING COMMENT '退配说明(大类)',
    return_reason_small     STRING COMMENT '退配原因(小类)',
    return_desc_small       STRING COMMENT '退配说明(小类)',
    sync_time               TIMESTAMP COMMENT '数据同步时间'
)
COMMENT '门店退货单'
partitioned by (dt STRING COMMENT '库存处理时间')
row format delimited fields terminated by ','
stored as orc
tblproperties ('orc.compress'='SNAPPY');
数据插入:

计算:时间字段 stock_deal_time

insert overwrite table dwd.dwd_order_store_return_to_vendor_i partition (dt)
select
    date_format(t.stock_deal_time,'yyyy-MM-dd') as trade_date
     ,dd.week_trade_date
     ,dd.month_trade_date
     ,hour(t.stock_deal_time) as hourly
     ,case when minute(t.stock_deal_time)>=0  and minute(t.stock_deal_time)<15 then 1
           when minute(t.stock_deal_time)>=15 and minute(t.stock_deal_time)<30 then 2
           when minute(t.stock_deal_time)>=30 and minute(t.stock_deal_time)<45 then 3
           when minute(t.stock_deal_time)>=45 and minute(t.stock_deal_time)<60 then 4
    end as `quarter`
     ,hour(t.stock_deal_time)*4 + case when minute(t.stock_deal_time)>=0  and minute(t.stock_deal_time)<15 then 1
                                       when minute(t.stock_deal_time)>=15 and minute(t.stock_deal_time)<30 then 2
                                       when minute(t.stock_deal_time)>=30 and minute(t.stock_deal_time)<45 then 3
                                       when minute(t.stock_deal_time)>=45 and minute(t.stock_deal_time)<60 then 4
    end  as quarters
     ,t.id
     ,t.uid
     ,t.order_id
     ,t.order_source

     ,t.store_no
     ,bs.store_name
     ,bs.store_sale_type
     ,bs.store_type_code
     ,bs.worker_num
     ,bs.store_area
     ,bs.city_id
     ,bs.city_name
     ,bs.region_code
     ,bs.region_name
     ,bs.is_day_clear

     ,t.goods_no
     ,t.goods_name
     ,t.dc_no
     ,t.dc_name
     ,t.vendor_no
     ,t.vendor_name
     ,t.return_price
     ,t.return_qty
     ,t.create_time
     ,t.stock_deal_time
     ,t.original_order_id
     ,t.is_fresh
     ,t.is_entity
     ,t.responsible_person
     ,t.return_reason_big
     ,t.return_desc_big
     ,t.return_reason_small
     ,t.return_desc_small
     ,t.sync_time

     ,date_format(t.stock_deal_time,'yyyy-MM-dd') as dt
from ods.ods_order_store_return_to_vendor_i t
inner join dim.dwd_dim_date_f as dd
 on date_format(t.stock_deal_time,'yyyy-MM-dd') = dd.trade_date
inner join dim.dwd_dim_store_i as bs
 on t.store_no=bs.store_no and bs.dt = '2023-11-23'
;

1.1.9 门店退配单

门店退给大仓的商品情况,包括单号、门店、时间、商品信息、数量、金额、原因等。

涉及表: ods_order_store_return_to_dc_i(门店退配单)、时间表、门店信息表

建表语句:
CREATE TABLE IF NOT EXISTS dwd.dwd_order_store_return_to_dc_i(
    trade_date              STRING COMMENT '库存处理日期',
    week_trade_date         STRING COMMENT '周一日期',
    month_trade_date        STRING COMMENT '月一日期',
    hourly                  BIGINT COMMENT '交易小时(0-23)',
    quarter                 BIGINT COMMENT '刻钟:1.0-15,2.15-30,3.30-45,4.45-60',
    quarters                BIGINT COMMENT '刻钟数:hourly*4+quarters',

    id                      BIGINT COMMENT '主键',
    uid                     STRING COMMENT '唯一标识',
    order_id                STRING COMMENT '退配单号',
    order_source            BIGINT COMMENT '下单来源,小程序、pc、智能补货、系统等',

    store_no                STRING COMMENT '门店编码',
    store_name              STRING COMMENT '门店名称',
    store_sale_type         BIGINT COMMENT '店铺销售类型',
    store_type_code         BIGINT COMMENT '分店类型',
    worker_num              BIGINT COMMENT '员工人数',
    store_area              DECIMAL(27, 2) COMMENT '门店面积',
    city_id                 BIGINT COMMENT '城市ID',
    city_name               STRING COMMENT '城市名称',
    region_code             STRING COMMENT '区域编码',
    region_name             STRING COMMENT '区域名称',
    is_day_clear            BIGINT COMMENT '是否日清:0否,1是',

    goods_no                STRING COMMENT '商品编码',
    goods_name              STRING COMMENT '商品名称',
    dc_no                   STRING COMMENT '配送中心编码',
    dc_name                 STRING COMMENT '配送中心名称',
    vendor_no               STRING COMMENT '供应商编码',
    vendor_name             STRING COMMENT '供应商名称',
    return_price            DECIMAL(27, 2) COMMENT '退配价',
    return_qty              DECIMAL(27, 3) COMMENT '退配数量',
    create_time             TIMESTAMP COMMENT '创建时间',
    stock_deal_time         TIMESTAMP COMMENT '库存处理时间',
    original_order_id       STRING COMMENT '退配原单号',
    is_fresh                BIGINT COMMENT '是否为生鲜店,0-否,1-是',
    is_entity               BIGINT COMMENT '是否实物退回,0-否,1-是',
    responsible_person      STRING COMMENT '责任归属方',
    return_reason_big       STRING COMMENT '退配原因(大类)',
    return_desc_big         STRING COMMENT '退配说明(大类)',
    return_reason_small     STRING COMMENT '退配原因(小类)',
    return_desc_small       STRING COMMENT '退配说明(小类)',
    sync_time               TIMESTAMP COMMENT '数据同步时间',
    batch_type_id           STRING COMMENT '批次类型id'
)
COMMENT '门店退配单'
partitioned by (dt STRING COMMENT '库存处理时间')
row format delimited fields terminated by ','
stored as orc
tblproperties ('orc.compress'='SNAPPY');
数据插入:

计算:时间字段 stock_deal_time

insert overwrite table dwd.dwd_order_store_return_to_dc_i partition (dt)
select
    date_format(t.stock_deal_time,'yyyy-MM-dd') as trade_date
     ,dd.week_trade_date
     ,dd.month_trade_date
     ,hour(t.stock_deal_time) as hourly
     ,case when minute(t.stock_deal_time)>=0  and minute(t.stock_deal_time)<15 then 1
           when minute(t.stock_deal_time)>=15 and minute(t.stock_deal_time)<30 then 2
           when minute(t.stock_deal_time)>=30 and minute(t.stock_deal_time)<45 then 3
           when minute(t.stock_deal_time)>=45 and minute(t.stock_deal_time)<60 then 4
    end as `quarter`
     ,hour(t.stock_deal_time)*4 + case when minute(t.stock_deal_time)>=0  and minute(t.stock_deal_time)<15 then 1
                                       when minute(t.stock_deal_time)>=15 and minute(t.stock_deal_time)<30 then 2
                                       when minute(t.stock_deal_time)>=30 and minute(t.stock_deal_time)<45 then 3
                                       when minute(t.stock_deal_time)>=45 and minute(t.stock_deal_time)<60 then 4
    end  as quarters

     ,t.id
     ,t.uid
     ,t.order_id
     ,t.order_source

     ,t.store_no
     ,bs.store_name
     ,bs.store_sale_type
     ,bs.store_type_code
     ,bs.worker_num
     ,bs.store_area
     ,bs.city_id
     ,bs.city_name
     ,bs.region_code
     ,bs.region_name
     ,bs.is_day_clear

     ,t.goods_no
     ,t.goods_name
     ,t.dc_no
     ,t.dc_name
     ,t.vendor_no
     ,t.vendor_name
     ,t.return_price
     ,t.return_qty
     ,t.create_time
     ,t.stock_deal_time
     ,t.original_order_id
     ,t.is_fresh
     ,t.is_entity
     ,t.responsible_person
     ,t.return_reason_big
     ,t.return_desc_big
     ,t.return_reason_small
     ,t.return_desc_small
     ,t.sync_time
     ,t.batch_type_id

     ,date_format(t.stock_deal_time,'yyyy-MM-dd') as dt
from ods.ods_order_store_return_to_dc_i t
inner join dim.dwd_dim_date_f as dd
 on date_format(t.stock_deal_time,'yyyy-MM-dd') = dd.trade_date
inner join dim.dwd_dim_store_i as bs
 on t.store_no = bs.store_no and bs.dt = '2023-11-23'
;

1.1.10 门店调入单

从其他门店调入本店的单据信息,包括时间、单号、商品、门店、商品信息、数量和金额等。

涉及表: ods_order_store_alloc_in_i(门店调入单) 、时间表、门店表

建表语句:
CREATE TABLE IF NOT EXISTS dwd.dwd_order_store_alloc_in_i(
    trade_date                  STRING COMMENT '库存处理日期',
    week_trade_date             STRING COMMENT '周一日期',
    month_trade_date            STRING COMMENT '月一日期',
    hourly                      BIGINT COMMENT '交易小时(0-23)',
    quarter                     BIGINT COMMENT '刻钟:1.0-15,2.15-30,3.30-45,4.45-60',
    quarters                    BIGINT COMMENT '刻钟数:hourly*4+quarters',

    id                          BIGINT COMMENT '主键',
    uid                         STRING COMMENT '唯一标识',
    order_id                    STRING COMMENT '调拨单号',
    order_source                BIGINT COMMENT '下单来源,小程序、PC等',

    goods_no                    STRING COMMENT '商品编码',
    goods_name                  STRING COMMENT '商品名称',

    alloc_in_store_no           STRING COMMENT '调入门店编码',
    alloc_in_store_name         STRING COMMENT '调入门店名称',
    alloc_in_store_sale_type    BIGINT COMMENT '调入店铺销售类型',
    alloc_in_store_type_code    BIGINT COMMENT '调入分店类型',
    alloc_in_worker_num         BIGINT COMMENT '调入门店员工人数',
    alloc_in_store_area         DECIMAL(27, 2) COMMENT '调入门店面积',
    alloc_in_city_id            BIGINT COMMENT '调入门店城市ID',
    alloc_in_city_name          STRING COMMENT '调入门店城市名称',
    alloc_in_region_code        STRING COMMENT '调入门店区域编码',
    alloc_in_region_name        STRING COMMENT '调入门店区域名称',
    alloc_in_is_day_clear       BIGINT COMMENT '调入门店是否日清:0否,1是',

    alloc_out_store_no          STRING COMMENT '调出门店编码',
    alloc_out_store_name        STRING COMMENT '调出门店名称',
    alloc_price                 DECIMAL(27, 2) COMMENT '调拨单价',
    alloc_qty                   DECIMAL(27, 3) COMMENT '调拨数量',
    alloc_reason                STRING COMMENT '调拨原因',
    alloc_amount                DECIMAL(27, 2) COMMENT '调拨金额',
    create_time                 TIMESTAMP COMMENT '创建时间',
    stock_deal_time             TIMESTAMP COMMENT '库存处理时间',
    sync_time                   TIMESTAMP COMMENT '数据同步时间',
    vendor_no                   STRING COMMENT '供应商编码',
    vendor_name                 STRING COMMENT '供应商名称'
)
COMMENT '门店调入单'
partitioned by (dt STRING COMMENT '库存处理时间')
row format delimited fields terminated by ','
stored as orc
tblproperties ('orc.compress'='SNAPPY');
数据插入:

计算:时间字段为stock_deal_time

insert overwrite table dwd.dwd_order_store_alloc_in_i partition (dt)
select
    date_format(t.stock_deal_time,'yyyy-MM-dd') as trade_date
     ,dd.week_trade_date
     ,dd.month_trade_date
     ,hour(t.stock_deal_time) as hourly
     ,case when minute(t.stock_deal_time)>=0  and minute(t.stock_deal_time)<15 then 1
           when minute(t.stock_deal_time)>=15 and minute(t.stock_deal_time)<30 then 2
           when minute(t.stock_deal_time)>=30 and minute(t.stock_deal_time)<45 then 3
           when minute(t.stock_deal_time)>=45 and minute(t.stock_deal_time)<60 then 4
    end as `quarter`
     ,hour(t.stock_deal_time)*4 + case when minute(t.stock_deal_time)>=0  and minute(t.stock_deal_time)<15 then 1
                                       when minute(t.stock_deal_time)>=15 and minute(t.stock_deal_time)<30 then 2
                                       when minute(t.stock_deal_time)>=30 and minute(t.stock_deal_time)<45 then 3
                                       when minute(t.stock_deal_time)>=45 and minute(t.stock_deal_time)<60 then 4
    end  as quarters

     ,t.id
     ,t.uid
     ,t.order_id
     ,t.order_source

     ,t.goods_no
     ,t.goods_name

     ,t.alloc_in_store_no
     ,bs.store_name as alloc_in_store_name
     ,bs.store_sale_type as alloc_in_store_sale_type
     ,bs.store_type_code as alloc_in_store_type_code
     ,bs.worker_num as alloc_in_worker_num
     ,bs.store_area as alloc_in_store_area
     ,bs.city_id as alloc_in_city_id
     ,bs.city_name as alloc_in_city_name
     ,bs.region_code as alloc_in_region_code
     ,bs.region_name as alloc_in_region_name
     ,bs.is_day_clear as alloc_in_is_clear

     ,t.alloc_out_store_no
     ,t.alloc_out_store_name
     ,t.alloc_price
     ,t.alloc_qty
     ,t.alloc_reason
     ,t.alloc_amount
     ,t.create_time
     ,t.stock_deal_time
     ,t.sync_time
     ,t.vendor_no
     ,t.vendor_name

     ,date_format(t.stock_deal_time,'yyyy-MM-dd') as dt
from ods.ods_order_store_alloc_in_i t
inner join dim.dwd_dim_date_f as dd
 on date_format(t.stock_deal_time,'yyyy-MM-dd') = dd.trade_date
inner join dim.dwd_dim_store_i as bs
 on t.alloc_in_store_no=bs.store_no and bs.dt = '2023-11-23'
;

1.1.11 门店调出单

从本店调入其他门店的单据信息,包括时间、单号、商品、门店、商品信息、数量和金额等。

涉及表: ods_order_store_alloc_out_i(门店调出单)、时间表、门店信息表

建表语句:
CREATE TABLE IF NOT EXISTS dwd.dwd_order_store_alloc_out_i(
    trade_date                   STRING COMMENT '库存处理日期',
    week_trade_date              STRING COMMENT '周一日期',
    month_trade_date             STRING COMMENT '月一日期',
    hourly                       BIGINT COMMENT '交易小时(0-23)',
    quarter                      BIGINT COMMENT '刻钟:1.0-15,2.15-30,3.30-45,4.45-60',
    quarters                     BIGINT COMMENT '刻钟数:hourly*4+quarters',

    id                           BIGINT COMMENT '主键',
    uid                          STRING COMMENT '唯一标识',
    order_id                     STRING COMMENT '调拨单号',
    order_source                 BIGINT COMMENT '下单来源,小程序、PC等',

    goods_no                     STRING COMMENT '商品编码',
    goods_name                   STRING COMMENT '商品名称',

    alloc_in_store_no            STRING COMMENT '调入门店编码',
    alloc_in_store_name          STRING COMMENT '调入门店名称',

    alloc_out_store_no           STRING COMMENT '调出门店编码',
    alloc_out_store_name         STRING COMMENT '调出门店名称',
    alloc_out_store_sale_type    BIGINT COMMENT '调出店铺销售类型',
    alloc_out_store_type_code    BIGINT COMMENT '调出分店类型',
    alloc_out_worker_num         BIGINT COMMENT '调出门店员工人数',
    alloc_out_store_area         DECIMAL(27, 2) COMMENT '调出门店面积',
    alloc_out_city_id            BIGINT COMMENT '调出门店城市ID',
    alloc_out_city_name          STRING COMMENT '调出门店城市名称',
    alloc_out_region_code        STRING COMMENT '调出门店区域编码',
    alloc_out_region_name        STRING COMMENT '调出门店区域名称',
    alloc_out_is_day_clear       BIGINT COMMENT '调出门店是否日清:0否,1是',

    alloc_price                  DECIMAL(27, 2) COMMENT '调拨单价',
    alloc_qty                    DECIMAL(27, 3) COMMENT '调拨数量',
    alloc_reason                 STRING COMMENT '调拨原因',
    alloc_amount                 DECIMAL(27, 2) COMMENT '调拨金额',
    create_time                  TIMESTAMP COMMENT '创建时间',
    stock_deal_time              TIMESTAMP COMMENT '库存处理时间',
    sync_time                    TIMESTAMP COMMENT '数据同步时间',
    vendor_no                    STRING COMMENT '供应商编码',
    vendor_name                  STRING COMMENT '供应商名称'
)
COMMENT '门店调出单'
partitioned by (dt STRING COMMENT '库存处理时间')
row format delimited fields terminated by ','
stored as orc
tblproperties ('orc.compress'='SNAPPY');
数据插入:

计算SQL: 时间字段为stock_deal_time

insert overwrite table dwd.dwd_order_store_alloc_out_i partition (dt)
select
    date_format(t.stock_deal_time,'yyyy-MM-dd') as trade_date
     ,dd.week_trade_date
     ,dd.month_trade_date
     ,hour(t.stock_deal_time) as hourly
     ,case when minute(t.stock_deal_time)>=0  and minute(t.stock_deal_time)<15 then 1
           when minute(t.stock_deal_time)>=15 and minute(t.stock_deal_time)<30 then 2
           when minute(t.stock_deal_time)>=30 and minute(t.stock_deal_time)<45 then 3
           when minute(t.stock_deal_time)>=45 and minute(t.stock_deal_time)<60 then 4
    end as `quarter`
     ,hour(t.stock_deal_time)*4 + case when minute(t.stock_deal_time)>=0  and minute(t.stock_deal_time)<15 then 1
                                       when minute(t.stock_deal_time)>=15 and minute(t.stock_deal_time)<30 then 2
                                       when minute(t.stock_deal_time)>=30 and minute(t.stock_deal_time)<45 then 3
                                       when minute(t.stock_deal_time)>=45 and minute(t.stock_deal_time)<60 then 4
    end  as quarters

     ,t.id
     ,t.uid
     ,t.order_id
     ,t.order_source

     ,t.goods_no
     ,t.goods_name

     ,t.alloc_in_store_no
     ,t.alloc_in_store_name

     ,t.alloc_out_store_no
     ,bs.store_name as alloc_out_store_name
     ,bs.store_sale_type as alloc_out_store_sale_type
     ,bs.store_type_code as alloc_out_store_type_code
     ,bs.worker_num as alloc_out_worker_num
     ,bs.store_area as alloc_out_store_area
     ,bs.city_id as alloc_out_city_id
     ,bs.city_name as alloc_out_city_name
     ,bs.region_code as alloc_out_region_code
     ,bs.region_name as alloc_out_region_name
     ,bs.is_day_clear as alloc_out_is_clear

     ,t.alloc_price
     ,t.alloc_qty
     ,t.alloc_reason
     ,t.alloc_amount
     ,t.create_time
     ,t.stock_deal_time
     ,t.sync_time
     ,t.vendor_no
     ,t.vendor_name

     ,date_format(t.stock_deal_time,'yyyy-MM-dd') as dt
from ods.ods_order_store_alloc_out_i t
inner join dim.dwd_dim_date_f as dd
 on date_format(t.stock_deal_time,'yyyy-MM-dd') = dd.trade_date
inner join dim.dwd_dim_store_i as bs
 on t.alloc_out_store_no=bs.store_no and bs.dt = '2023-11-23'
;

1.1.12 门店要货单

门店的要货信息,包括时间、单号、门店、商品信息、数量和金额、要货方式、状态等。

涉及表: ods_order_store_require_i(门店要货单)和日期表和门店表

建表语句:
CREATE TABLE IF NOT EXISTS dwd.dwd_order_store_require_i(
    trade_date               STRING COMMENT '确认日期',
    week_trade_date          STRING COMMENT '周一日期',
    month_trade_date         STRING COMMENT '月一日期',
    hourly                   BIGINT COMMENT '交易小时(0-23)',
    quarter                  BIGINT COMMENT '刻钟:1.0-15,2.15-30,3.30-45,4.45-60',
    quarters                 BIGINT COMMENT '刻钟数:hourly*4+quarters',

    id                       BIGINT COMMENT '主键',
    uid                      STRING COMMENT '唯一标识',
    order_id                 STRING COMMENT '要货单号',
    order_source             BIGINT COMMENT '下单来源,小程序、pc、智能补货、系统等',

    store_no                 STRING COMMENT '门店编码',
    store_name               STRING COMMENT '门店名称',
    store_sale_type          BIGINT COMMENT '店铺销售类型',
    store_type_code          BIGINT COMMENT '分店类型',
    worker_num               BIGINT COMMENT '员工人数',
    store_area               DECIMAL(27, 2) COMMENT '门店面积',
    city_id                  BIGINT COMMENT '城市ID',
    city_name                STRING COMMENT '城市名称',
    region_code              STRING COMMENT '区域编码',
    region_name              STRING COMMENT '区域名称',
    is_day_clear             BIGINT COMMENT '是否日清:0否,1是',

    goods_no                 STRING COMMENT '商品编码',
    goods_name               STRING COMMENT '商品名称',
    dc_no                    STRING COMMENT '配送中心编码',
    dc_name                  STRING COMMENT '配送中心名称',
    vendor_no                STRING COMMENT '供应商编码',
    vendor_name              STRING COMMENT '供应商名称',
    group_no                 STRING COMMENT '采购柜组编号',
    require_price            DECIMAL(27, 2) COMMENT '要货价格',
    require_qty              DECIMAL(27, 3) COMMENT '要货数量',
    create_time              TIMESTAMP COMMENT '创建时间',
    send_time                TIMESTAMP COMMENT '预计送货时间',
    collect_require_order_id STRING COMMENT '要货汇总单号',
    require_type_code        BIGINT COMMENT '要货类型:1-直送,2-配送,3-代发',
    is_online                BIGINT COMMENT '1-线上,0-线下',
    confirm_time             TIMESTAMP COMMENT '审核时间',
    is_canceled              BIGINT COMMENT '1-取消,0-正常',
    sync_time                TIMESTAMP COMMENT '数据同步时间',
    is_urgent                BIGINT COMMENT '是否加急 0,否 1,是',
    original_order_price     DECIMAL(27, 2) COMMENT '原单价'
)
COMMENT '门店要货单'
partitioned by (dt STRING COMMENT 'confirm_time时间')
row format delimited fields terminated by ','
stored as orc
tblproperties ('orc.compress'='SNAPPY');
数据插入:

计算: 时间字段为confirm_time

insert overwrite table dwd.dwd_order_store_require_i partition (dt)
select
    date_format(t.confirm_time,'yyyy-MM-dd') as trade_date
     ,dd.week_trade_date
     ,dd.month_trade_date
     ,hour(t.confirm_time) as hourly
     ,case when minute(t.confirm_time)>=0  and minute(t.confirm_time)<15 then 1
           when minute(t.confirm_time)>=15 and minute(t.confirm_time)<30 then 2
           when minute(t.confirm_time)>=30 and minute(t.confirm_time)<45 then 3
           when minute(t.confirm_time)>=45 and minute(t.confirm_time)<60 then 4
    end as `quarter`
     ,hour(t.confirm_time)*4 + case when minute(t.confirm_time)>=0  and minute(t.confirm_time)<15 then 1
                                    when minute(t.confirm_time)>=15 and minute(t.confirm_time)<30 then 2
                                    when minute(t.confirm_time)>=30 and minute(t.confirm_time)<45 then 3
                                    when minute(t.confirm_time)>=45 and minute(t.confirm_time)<60 then 4
    end  as quarters
     ,t.id
     ,t.uid
     ,t.order_id
     ,t.order_source

     ,t.store_no
     ,bs.store_name
     ,bs.store_sale_type
     ,bs.store_type_code
     ,bs.worker_num
     ,bs.store_area
     ,bs.city_id
     ,bs.city_name
     ,bs.region_code
     ,bs.region_name
     ,bs.is_day_clear

     ,t.goods_no
     ,t.goods_name
     ,t.dc_no
     ,t.dc_name
     ,t.vendor_no
     ,t.vendor_name
     ,t.group_no
     ,t.require_price
     ,t.require_qty
     ,t.create_time
     ,t.send_time
     ,t.collect_require_order_id
     ,t.require_type_code
     ,t.is_online
     ,t.confirm_time
     ,t.is_canceled
     ,t.sync_time
     ,t.is_urgent
     ,t.original_order_price

     ,date_format(t.confirm_time,'yyyy-MM-dd') as dt
from ods.ods_order_store_require_i t
inner join dim.dwd_dim_date_f as dd
 on date_format(t.confirm_time,'yyyy-MM-dd') = dd.trade_date
inner join dim.dwd_dim_store_i as bs
 on t.store_no=bs.store_no and bs.dt = '2023-11-23'
;

1.2 DWM层

数据仓库中间层: ==Data Warehouse Middle==

为DWS层准备数据,提前合并部分数据

注意:具体计算只计算增量,全量计算直接去掉where条件即可。或者在调度时使用补数的方法,将历史分区一天一天跑出来即可!

1.2.1 商品销售明细(核销)

dwm_sold_goods_sold_dtl_i作为核销主题的小宽表,要能涵盖线上线下各渠道的销售数据。这个逻辑比较简单,将dwd_sale_store_sale_dtl_i与dwd_sold_shop_order_dtl_i两表进行合并即可。需要注意的就是字段要对齐,对于补充的字段要按照定义进行补充。比如 0 as is_online_order,4 as order_type ,4 as express_type 等。

建表语句:
CREATE TABLE IF NOT EXISTS dwm.dwm_sold_goods_sold_dtl_i(
    trade_date_time             STRING COMMENT '核销时间',
    trade_date                  STRING COMMENT '交易日期',
    week_trade_date             STRING COMMENT '周一日期',
    month_trade_date            STRING COMMENT '月一日期',
    hourly                      BIGINT COMMENT '交易小时(0-23)',
    quarter                     BIGINT COMMENT '刻钟:1.0-15,2.15-30,3.30-45,4.45-60',
    quarters                    BIGINT COMMENT '刻钟数:hourly*4+quarters',

    parent_store_no             STRING COMMENT '母店编码',
    store_no                    STRING COMMENT '店铺编码',
    store_name                  STRING COMMENT '店铺名称',
    store_sale_type             BIGINT COMMENT '店铺销售类型',
    store_type_code             BIGINT COMMENT '分店类型',
    worker_num                  BIGINT COMMENT '员工人数',
    store_area                  DECIMAL(27, 2) COMMENT '门店面积',
    city_id                     BIGINT COMMENT '城市ID',
    city_name                   STRING COMMENT '城市名称',
    region_code                 STRING COMMENT '区域编码',
    region_name                 STRING COMMENT '区域名称',
    is_day_clear                BIGINT COMMENT '是否日清:0否,1是',

    trade_type                  BIGINT COMMENT '结算类型(0.正常交易,1.赠品发放,2.退货,4.培训,5.取消交易)',
    source_type                 BIGINT COMMENT '交易来源1:线下POS;2:三方平台;3:传智鲜商城;4:黑马优选团;5:传智大客户;6:传智其他;7:黑马优选;8:优选海淘;9:优选大客户;10:优选POS;11:优选APP;12:优选H5;13:店长工具线下;14:店长工具线上;15:黑马其他',
    source_type_name            STRING COMMENT '交易来源名称',
    sale_type                   BIGINT COMMENT '销售类型 1.实物,2.代客,3.优选小程序,4.离店,5.传智鲜小程序,6.第三方平台,7.其他,8.大客户',
    is_online_order             BIGINT COMMENT '是否为线上单:0否,1是',
    member_type                 BIGINT COMMENT '会员类型:0非会员,1线上会员,2实体卡会员',
    is_balance_consume          BIGINT COMMENT '是否有余额支付:0否,1是',
    order_type                  BIGINT COMMENT '配送类型(真正的订单类型由业务类型来决定):1-及时送;2-隔日送;3-自提单;4-线下单',
    express_type                BIGINT COMMENT '配送方式:0-三方平台配送;1-自配送;2-快递;3-自提;4-线下',

    parent_order_no             STRING COMMENT '母订单编号',
    order_no                    STRING COMMENT '订单编号',

    create_time                 STRING COMMENT '创建时间',
    is_cancel                   BIGINT COMMENT '是否取消',
    cancel_time                 STRING COMMENT '取消时间',
    last_update_time            STRING COMMENT 'pos_sale表最后一次更新时间',

    zt_id                       BIGINT COMMENT '中台ID',
    member_id                   BIGINT COMMENT '会员ID',
    card_no                     STRING COMMENT '卡号',

    share_user_id               STRING COMMENT '分享人用户ID',
    commission_amount           DECIMAL(27, 2) COMMENT '佣金',
    is_tuan_head                BIGINT COMMENT '是否为团长订单',
    store_leader_id             BIGINT COMMENT '团长id',
    order_group_no              STRING COMMENT '团单号',

    first_category_no           STRING COMMENT '一级分类编码',
    first_category_name         STRING COMMENT '一级分类名称',
    second_category_no          STRING COMMENT '二级分类编码',
    second_category_name        STRING COMMENT '二级分类名称',
    third_category_no           STRING COMMENT '三级分类编码',
    third_category_name         STRING COMMENT '三级分类名称',
    goods_no                    STRING COMMENT '商品编码',
    goods_name                  STRING COMMENT '商品名称',

    supply_team                 BIGINT COMMENT '供应链团队 1.平台商品,2.优选标品,3.传智鲜标品,4.传智鲜生鲜,5优选POS商品',
    dc_no                       STRING COMMENT '采购仓库编号',
    dc_name                     STRING COMMENT '采购仓库名称',
    group_no                    STRING COMMENT '采购柜组编号',
    group_name                  STRING COMMENT '采购柜组名称',
    trade_mode_id               BIGINT COMMENT '结算方式:1购销,2联营',
    vendor_id                   BIGINT COMMENT '供应商ID',
    contract_no                 STRING COMMENT '合同编号',
    is_clean                    BIGINT COMMENT '商品是否日清:0否,1是',
    is_daily_clear              BIGINT COMMENT '商品是否参加日清活动:0否,1是',

    sale_qty                    DECIMAL(27, 3) COMMENT '商品销售数量',
    sale_amount                 DECIMAL(27, 2) COMMENT '商品销售金额',
    dis_amount                  DECIMAL(27, 2) COMMENT '商品折扣金额',
    sale_cost                   DECIMAL(27, 2) COMMENT '商品销售成本',
    balance_amount              DECIMAL(27, 2) COMMENT '余额支付',

    order_total_amount          DECIMAL(27, 2) COMMENT '订单总金额(平摊)',
    order_discount_amount       DECIMAL(27, 2) COMMENT '订单优惠金额=商家承担优惠金额+平台补贴金额(平摊)',
    order_paid_amount           DECIMAL(27, 2) COMMENT '实付金额(平摊)'
)
COMMENT '商品销售明细(核销)'
partitioned by(dt STRING COMMENT '核销日期')
row format delimited fields terminated by ','
stored as orc
tblproperties ('orc.compress'='SNAPPY');
处理说明:
1.dwd.dwd_sale_store_sale_dtl_i(门店销售明细宽表):
	1- 新增is_online_order(是否为线上单), 设置值为0
	2- 新增order_type(配送类型), 设置值为4
	3- 新增express_type(配送方式), 设置值为4
	4- 新增is_cancel(是否取消), 判断: 根据trade_type(结算类型)当为5返回1, 否则返回0
	5- 新增cancel_time(取消时间),判断: 当 trade_type(结算类型)为5返回last_update_time,否则为空
	6- 新增is_tuan_head(是否为团长订单), 设置值为0
	7- 新增store_leader_id(团长ID) , 设置为0
	8- 新增order_group_no(团单号), 设置为0
	9- 新增order_total_amount(订单总金额(平摊))、order_discount_amount(订单优惠金额)、order_paid_amount(实付金额(平摊)) 分别对应的值为 sale_amount、dis_amount、sale_amount



2.dwd.dwd_sold_shop_order_dtl_i(商城核销明细表):
	1- 修改 source_type, 由原来的订单来源, 变更为交易来源: 需要对应值重新映射
	2- 新增 source_type_name(交易来源名称), 设置对应编号的对应值即可
	3- 修改 sale_type为销售类型, 基于 原表中source_type
	4- order_no(订单编号)为order_id
	5- member_id(会员ID)为buyer_id
	6- 新增 card_no(卡号), 值为空
	7- 新增 share_user_id(分享人用户ID), 值为0
	8- 新增is_clean(商品是否日清:0否,1是) 和 is_daily_clear(商品是否参加日清活动:0否,1是) 值为0
	9- create_time、cancel_time、last_update_time 将三个时间戳类型 转换为字符串
	10- 类型调整: order_group_no 调整为int类型
	
	
3.上述两个表union all即可

数据插入:
-- DWM层: 销售明细宽表(将线上和线下的两部分数据进行union all 合并)
insert overwrite table dwm.dwm_sold_goods_sold_dtl_i partition(dt)
select
    trade_date_time,
    trade_date,
    week_trade_date,
    month_trade_date,
    hourly,
    quarter,
    quarters,
    parent_store_no,
    store_no,
    store_name,
    store_sale_type,
    store_type_code,
    worker_num,
    store_area,
    city_id,
    city_name,
    region_code,
    region_name,
    is_day_clear,
    trade_type,
    source_type,
    source_type_name,
    sale_type,
    0 as is_online_order,
    member_type,
    is_balance_consume,
    4 as order_type,
    4 as express_type,
    parent_order_no,
    order_no,
    trade_date_time as create_time,
    if(
        trade_type = 5,
        1,
        0
    ) as is_cancel,
    if(
        trade_type = 5,
        last_update_time,
        ''
    ) as cancel_time,
    last_update_time,
    zt_id,
    member_id,
    card_no,
    cast(share_user_id as int) as share_user_id,
    commission_amount,
    0 as is_tuan_head,
    0 as store_leader_id,
    '0' as order_group_no,
    first_category_no,
    first_category_name,
    second_category_no,
    second_category_name,
    third_category_no,
    third_category_name,
    goods_no,
    goods_name,
    supply_team,
    dc_no,
    dc_name,
    group_no,
    group_name,
    trade_mode_id,
    vendor_id,
    contract_no,
    is_clean,
    is_daily_clear,
    sale_qty,
    sale_amount,
    dis_amount,
    sale_cost,
    balance_amount,
    sale_amount as order_total_amount,
    dis_amount as order_discount_amount,
    sale_amount as order_paid_amount,
    dt
from dwd.dwd_sale_store_sale_dtl_i  -- 后续增量加上 where  dt = '昨天'

union all

select
    cast(complete_time as timestamp) as trade_date_time,
    cast(trade_date as timestamp) as trade_date,
    week_trade_date,
    month_trade_date,
    hourly,
    quarter,
    quarters,
    store_no as parent_store_no,
    store_no,
    store_name,
    store_sale_type,
    store_type_code,
    worker_num,
    store_area,
    city_id,
    city_name,
    region_code,
    region_name,
    is_day_clear,
    trade_type,
    if(
        source_type in(10,20,30,40,41,70),
        2,
        if(
            source_type = 50,
            3,
            7
        )
    ) as source_type,
    if(
       source_type in(10,20,30,40,41,70),
        '三方平台',
        if(
            source_type = 50,
            '传智鲜商城',
            '黑马优选'
        )
    ) as source_type_name,
    if(
        source_type in(10,20,30,40,41,70),
        6,
        if(
            source_type = 50,
            5,
            3
        )

    ) as sale_type,
    1 as is_online_order,
    1 as member_type,
    is_balance_consume,
    order_type,
    express_type,
    parent_order_no,
    order_id as order_no,
    create_time,
    is_cancel,
    date_format(cancel_time,'yyyy-MM-dd HH:mm:ss') as cancel_time,
    date_format(last_update_time,'yyyy-MM-dd HH:mm:ss') as last_update_time,
    zt_id,
    buyer_id as member_id,
    '' as card_no,
    0 as share_user_id,
    commission_amount,
    is_tuan_head,
    store_leader_id,
    order_group_no,
    first_category_no,
    first_category_name,
    second_category_no,
    second_category_name,
    third_category_no,
    third_category_name,
    goods_no,
    goods_name,
    supply_team,
    dc_no,
    dc_name,
    group_no,
    group_name,
    trade_mode_id,
    vendor_id,
    contract_no,
    0 as is_clean,
    0 as is_daily_clear,
    sale_qty,
    sale_amount,
    dis_amount,
    sale_cost,
    balance_amount,
    order_total_amount,
    order_discount_amount,
    order_paid_amount,
    dt
from dwd.dwd_sold_shop_order_dtl_i;

1.2.2 门店商品损耗刻表

门店商品损耗刻表,粒度为门店商品刻,包含时间、门店、商品、品类等信息,便于dws/ads层使用。

建表语句:
CREATE TABLE IF NOT EXISTS dwm.dwm_stock_store_goods_loss_quarter_i(
    trade_date              STRING COMMENT '交易日期',
    week_trade_date         STRING COMMENT '周一日期',
    month_trade_date        STRING COMMENT '月一日期',
    hourly                  BIGINT COMMENT '交易小时(0-23)',
    quarter                 BIGINT COMMENT '刻钟:1.0-15,2.15-30,3.30-45,4.45-60',
    quarters                BIGINT COMMENT '刻钟数:hourly*4+quarters',
	
    store_no                STRING COMMENT '店铺编码',
    store_name              STRING COMMENT '店铺名称',
    store_sale_type         BIGINT COMMENT '店铺销售类型',
    store_type_code         BIGINT COMMENT '分店类型',
    worker_num              BIGINT COMMENT '员工人数',
    store_area              DECIMAL(27, 2) COMMENT '门店面积',
    city_id                 BIGINT COMMENT '城市ID',
    city_name               STRING COMMENT '城市名称',
    region_code             STRING COMMENT '区域编码',
    region_name             STRING COMMENT '区域名称',
    is_day_clear            BIGINT COMMENT '是否日清:0否,1是',
    -- 商品表
    first_category_no       STRING COMMENT '一级分类编码',
    first_category_name     STRING COMMENT '一级分类名称',
    second_category_no      STRING COMMENT '二级分类编码',
    second_category_name    STRING COMMENT '二级分类名称',
    third_category_no       STRING COMMENT '三级分类编码',
    third_category_name     STRING COMMENT '三级分类名称',
    
    goods_no                STRING COMMENT '商品编码',
    -- 商品表
    goods_name              STRING COMMENT '商品名称',
    -- 门店日清商品表
    is_clean                BIGINT COMMENT '商品是否日清',

    loss_qty                DECIMAL(27, 3) COMMENT '损耗数量',
    loss_amount             DECIMAL(27, 2) COMMENT '损耗金额'
)
COMMENT '门店商品损耗刻表'
partitioned by(dt STRING COMMENT '库存处理时间')
row format delimited fields terminated by ','
stored as orc
tblproperties ('orc.compress'='SNAPPY');
处理说明:

基于dwd_stock_store_stock_adj_i表进行计算,因为dwd_stock_store_stock_adj_i是明细表,需要进行聚合操作。聚合之后再关联维表进行拉宽,可以提高效率。

需求说明:
	根据维表表以外的其他的字段进行分组, 聚合损耗数量和损耗金额
	损耗类别: adj_type_big in ('日清', '报损/溢', '人工盘点', '盘点更正', '周清')
数据插入:
-- 计算损耗
with t1 as (
    select
        trade_date,
        week_trade_date,
        month_trade_date,
        hourly,
        quarter,
        quarters,
        store_no,
        store_name,
        store_sale_type,
        store_type_code,
        worker_num,
        store_area,
        city_id,
        city_name,
        region_code,
        region_name,
        is_day_clear,
        goods_no,

        sum(adj_qty) as loss_qty,
        sum(adj_amount) as loss_amount
    from dwd.dwd_stock_store_stock_adj_i where adj_type_big in ('日清','报损/溢','人工盘点','盘点更正','周清')
    group by
        trade_date,
        week_trade_date,
        month_trade_date,
        hourly,
        quarter,
        quarters,
        store_no,
        store_name,
        store_sale_type,
        store_type_code,
        worker_num,
        store_area,
        city_id,
        city_name,
        region_code,
        region_name,
        is_day_clear,
        goods_no
)
insert overwrite table dwm.dwm_stock_store_goods_loss_quarter_i partition (dt)
select
    t1.trade_date,
    t1.week_trade_date,
    t1.month_trade_date,
    t1.hourly,
    t1.quarter,
    t1.quarters,
    t1.store_no,
    t1.store_name,
    t1.store_sale_type,
    t1.store_type_code,
    t1.worker_num,
    t1.store_area,
    t1.city_id,
    t1.city_name,
    t1.region_code,
    t1.region_name,
    t1.is_day_clear,
    t2.first_category_no,
    t2.first_category_name,
    t2.second_category_no,
    t2.second_category_name,
    t2.third_category_no,
    t2.third_category_name,
    t1.goods_no,
    t2.goods_name,
    t2.is_clear as is_clean,
    t1.loss_qty,
    t1.loss_amount,
    trade_date as dt
from  t1 left join dim.dwd_dim_store_goods_i t2
    on t1.store_no = t2.store_no and t1.goods_no = t2.goods_no and t2.dt = '2023-11-23';

1.2.3 门店商品收货刻表

净收货=收货-退货-退配+调入-调出,所以需要综合五张dwd的order主题的表。这里为了便于其他分析,不仅存净收货指标,也把收货、退货、退配、调入、调出这几个指标分别存上。

建表语句:
CREATE TABLE IF NOT EXISTS dwm.dwm_order_store_goods_receipt_quarter_i(
    trade_date                  STRING COMMENT '交易日期',
    week_trade_date             STRING COMMENT '周一日期',
    month_trade_date            STRING COMMENT '月一日期',
    hourly                      BIGINT COMMENT '交易小时(0-23)',
    quarter                     BIGINT COMMENT '刻钟:1.0-15,2.15-30,3.30-45,4.45-60',
    quarters                    BIGINT COMMENT '刻钟数:hourly*4+quarters',

    store_no                    STRING COMMENT '店铺编码',
    -- 门店信息表
    store_name                  STRING COMMENT '店铺名称',
    store_sale_type             BIGINT COMMENT '店铺销售类型',
    store_type_code             BIGINT COMMENT '分店类型',
    worker_num                  BIGINT COMMENT '员工人数',
    store_area                  DECIMAL(27, 2) COMMENT '门店面积',
    city_id                     BIGINT COMMENT '城市ID',
    city_name                   STRING COMMENT '城市名称',
    region_code                 STRING COMMENT '区域编码',
    region_name                 STRING COMMENT '区域名称',
    is_day_clear                BIGINT COMMENT '是否日清:0否,1是',
	-- 商品表
    first_category_no           STRING COMMENT '一级分类编码',
    first_category_name         STRING COMMENT '一级分类名称',
    second_category_no          STRING COMMENT '二级分类编码',
    second_category_name        STRING COMMENT '二级分类名称',
    third_category_no           STRING COMMENT '三级分类编码',
    third_category_name         STRING COMMENT '三级分类名称',
    
    goods_no                    STRING COMMENT '商品编码',
    -- 商品表
    goods_name                  STRING COMMENT '商品名称',
    -- 门店日清商品表
    is_clean                    BIGINT COMMENT '商品是否日清',

    receive_qty                 DECIMAL(27, 3) COMMENT '收货数量(从供应商或者仓库)',
    receive_amount              DECIMAL(27, 2) COMMENT '收货金额',
    return_vendor_qty           DECIMAL(27, 3) COMMENT '退货数量(退给供应商)',
    return_vendor_amount        DECIMAL(27, 2) COMMENT '退货金额',
    return_dc_qty               DECIMAL(27, 3) COMMENT '退配数量(退给仓库)',
    return_dc_amount            DECIMAL(27, 2) COMMENT '退配金额',
    allocation_in_qty           DECIMAL(27, 3) COMMENT '调入数量',
    allocation_in_amount        DECIMAL(27, 2) COMMENT '调入金额',
    allocation_out_qty          DECIMAL(27, 3) COMMENT '调出数量',
    allocation_out_amount       DECIMAL(27, 2) COMMENT '调出金额',
    receipt_qty                 DECIMAL(27, 3) COMMENT '净收货数量',
    receipt_amount              DECIMAL(27, 2) COMMENT '净收货金额(收货-退货-退配+调入-调出)',
    receipt_cost                DECIMAL(27, 2) COMMENT '净收货成本(总额95%)'
)
COMMENT '门店商品收货刻表'
partitioned by(dt STRING COMMENT '收货日期')
row format delimited fields terminated by ','
stored as orc
tblproperties ('orc.compress'='SNAPPY');
处理说明:

五张dwd表进行union,然后进行group by即可。需要注意的是,要看清源表的数据的正负,比如退货和退配,本身就是负值,此时直接加减即可。但是调入调出就需要变一下符号。

第一步: 将收货 退货 退配 调入 调出 分别获取出来

第二步: 将五个表union all,生成临时结果集t1

第三步: 执行group by 操作计算最终的收货 退货 退配 调入 调出 相关值,生成临时结果集t2

第四步: 计算最终的净收货,插入到dwm_order_store_goods_receipt_quarter_i表中

数据插入:
with t1 as (
    select
        trade_date,
        week_trade_date,
        month_trade_date,
        hourly,
        quarter,
        quarters,
        store_no,
        store_name,
        store_sale_type,
        store_type_code,
        worker_num,
        store_area,
        city_id,
        city_name,
        region_code,
        region_name,
        is_day_clear,
        goods_no,
        receive_qty,
        receive_price as receive_amount,
        0 as return_vendor_qty,
        0 as return_vendor_amount,
        0 as return_dc_qty,
        0 as return_dc_amount,
        0 as allocation_in_qty,
        0 as allocation_in_amount,
        0 as allocation_out_qty,
        0 as allocation_out_amount

    from dwd.dwd_order_store_receive_i

    union all
    
    select
        trade_date,
        week_trade_date,
        month_trade_date,
        hourly,
        quarter,
        quarters,
        store_no,
        store_name,
        store_sale_type,
        store_type_code,
        worker_num,
        store_area,
        city_id,
        city_name,
        region_code,
        region_name,
        is_day_clear,
        goods_no,
        0 as receive_qty,
        0 as receive_amount,
        return_qty as return_vendor_qty,
        return_price as return_vendor_amount,
        0 as return_dc_qty,
        0 as return_dc_amount,
        0 as allocation_in_qty,
        0 as allocation_in_amount,
        0 as allocation_out_qty,
        0 as allocation_out_amount

    from dwd.dwd_order_store_return_to_vendor_i
    
    union all
    
    select
        trade_date,
        week_trade_date,
        month_trade_date,
        hourly,
        quarter,
        quarters,
        store_no,
        store_name,
        store_sale_type,
        store_type_code,
        worker_num,
        store_area,
        city_id,
        city_name,
        region_code,
        region_name,
        is_day_clear,
        goods_no,
        0 as receive_qty,
        0 as receive_amount,
        0 as return_vendor_qty,
        0 as return_vendor_amount,
        return_qty as return_dc_qty,
        return_price as return_dc_amount,
        0 as allocation_in_qty,
        0 as allocation_in_amount,
        0 as allocation_out_qty,
        0 as allocation_out_amount

    from dwd.dwd_order_store_return_to_dc_i
    
    union all
    
    select
        trade_date,
        week_trade_date,
        month_trade_date,
        hourly,
        quarter,
        quarters,
        alloc_in_store_no as store_no,
        alloc_in_store_name as store_name,
        alloc_in_store_sale_type as store_sale_type,
        alloc_in_store_type_code as store_type_code,
        alloc_in_worker_num as worker_num,
        alloc_in_store_area as store_area,
        alloc_in_city_id as city_id,
        alloc_in_city_name as city_name,
        alloc_in_region_code as region_code,
        alloc_in_region_name as region_name,
        alloc_in_is_day_clear as is_day_clear,
        goods_no,
        0 as receive_qty,
        0 as receive_amount,
        0 as return_vendor_qty,
        0 as return_vendor_amount,
        0 as return_dc_qty,
        0 as return_dc_amount,
        alloc_qty as allocation_in_qty,
        alloc_amount as allocation_in_amount,
        0 as allocation_out_qty,
        0 as allocation_out_amount

    from dwd.dwd_order_store_alloc_in_i
    
    union all
    
    select
        trade_date,
        week_trade_date,
        month_trade_date,
        hourly,
        quarter,
        quarters,
        alloc_out_store_no as store_no,
        alloc_out_store_name as store_name,
        alloc_out_store_sale_type as store_sale_type,
        alloc_out_store_type_code as store_type_code,
        alloc_out_worker_num as worker_num,
        alloc_out_store_area as store_area,
        alloc_out_city_id as city_id,
        alloc_out_city_name as city_name,
        alloc_out_region_code as region_code,
        alloc_out_region_name as region_name,
        alloc_out_is_day_clear as is_day_clear,
        goods_no,
        0 as receive_qty,
        0 as receive_amount,
        0 as return_vendor_qty,
        0 as return_vendor_amount,
        0 as return_dc_qty,
        0 as return_dc_amount,
        0 as allocation_in_qty,
        0 as allocation_in_amount,
        alloc_qty as allocation_out_qty,
        alloc_amount as allocation_out_amount

    from dwd.dwd_order_store_alloc_out_i
),
t2 as (
    select
        trade_date,
        week_trade_date,
        month_trade_date,
        hourly,
        quarter,
        quarters,
        store_no,
        store_name,
        store_sale_type,
        store_type_code,
        worker_num,
        store_area,
        city_id,
        city_name,
        region_code,
        region_name,
        is_day_clear,
        goods_no,
        sum(receive_qty) as receive_qty,
        sum(receive_qty * receive_amount) as receive_amount,
        sum(return_vendor_qty) as return_vendor_qty,
        sum(return_vendor_qty*return_vendor_amount) as return_vendor_amount,
        sum(return_dc_qty) as return_dc_qty,
        sum(return_dc_qty * return_dc_amount) as return_dc_amount,
        sum(allocation_in_qty) as allocation_in_qty,
        sum(allocation_in_amount) as allocation_in_amount,
        sum(allocation_out_qty) as allocation_out_qty,
        sum(allocation_out_amount) as allocation_out_amount
    from t1
    group by
        trade_date,
        week_trade_date,
        month_trade_date,
        hourly,
        quarter,
        quarters,
        store_no,
        store_name,
        store_sale_type,
        store_type_code,
        worker_num,
        store_area,
        city_id,
        city_name,
        region_code,
        region_name,
        is_day_clear,
        goods_no
)
insert overwrite table dwm.dwm_order_store_goods_receipt_quarter_i partition (dt)
select
    t2.trade_date,
    t2.week_trade_date,
    t2.month_trade_date,
    t2.hourly,
    t2.quarter,
    t2.quarters,
    t2.store_no,
    t2.store_name,
    t2.store_sale_type,
    t2.store_type_code,
    t2.worker_num,
    t2.store_area,
    t2.city_id,
    t2.city_name,
    t2.region_code,
    t2.region_name,
    t2.is_day_clear,
    t3.first_category_no,
    t3.first_category_name,
    t3.second_category_no,
    t3.second_category_name,
    t3.third_category_no,
    t3.third_category_name,
    t2.goods_no,
    t3.goods_name,
    t3.is_clear as is_clean,
    t2.receive_qty,
    t2.receive_amount,

    t2.return_vendor_qty,
    t2.return_vendor_amount,

    t2.return_dc_qty,
    t2.return_dc_amount,

    t2.allocation_in_qty,
    t2.allocation_in_amount,

    t2.allocation_out_qty,
    t2.allocation_out_amount,

    t2.receive_qty + return_vendor_qty + return_dc_qty + allocation_in_qty + allocation_out_qty  as receipt_qty,
    t2.receive_amount + t2.return_vendor_amount + t2.return_dc_amount + t2.allocation_in_amount + t2.allocation_out_amount as receipt_amount,
    (t2.receive_amount + t2.return_vendor_amount + t2.return_dc_amount + t2.allocation_in_amount + t2.allocation_out_amount) * 0.95 as receipt_cost,
    t2.trade_date  as dt
from t2 left join dim.dwd_dim_store_goods_i t3 on t3.dt = '2023-11-23' and t2.store_no = t3.store_no and t2.goods_no = t3.goods_no;

1.2.4 门店商品要货刻表

基于dwd_order_store_require_i进行聚合操作。聚合之后再关联维表进行拉宽,可以提高效率。

建表语句:
CREATE TABLE IF NOT EXISTS dwm.dwm_order_store_goods_require_quarter_i(
    trade_date              STRING COMMENT '交易日期',
    week_trade_date         STRING COMMENT '周一日期',
    month_trade_date        STRING COMMENT '月一日期',
    hourly                  BIGINT COMMENT '交易小时(0-23)',
    quarter                 BIGINT COMMENT '刻钟:1.0-15,2.15-30,3.30-45,4.45-60',
    quarters                BIGINT COMMENT '刻钟数:hourly*4+quarters',

    store_no                STRING COMMENT '店铺编码',
    store_name              STRING COMMENT '店铺名称',
    store_sale_type         BIGINT COMMENT '店铺销售类型',
    store_type_code         BIGINT COMMENT '分店类型',
    worker_num              BIGINT COMMENT '员工人数',
    store_area              DECIMAL(27, 2) COMMENT '门店面积',
    city_id                 BIGINT COMMENT '城市ID',
    city_name               STRING COMMENT '城市名称',
    region_code             STRING COMMENT '区域编码',
    region_name             STRING COMMENT '区域名称',
    is_day_clear            BIGINT COMMENT '是否日清:0否,1是',
	-- 商品表
    first_category_no       STRING COMMENT '一级分类编码',
    first_category_name     STRING COMMENT '一级分类名称',
    second_category_no      STRING COMMENT '二级分类编码',
    second_category_name    STRING COMMENT '二级分类名称',
    third_category_no       STRING COMMENT '三级分类编码',
    third_category_name     STRING COMMENT '三级分类名称',
    
    goods_no                STRING COMMENT '商品编码',
    -- 商品表
    goods_name              STRING COMMENT '商品名称',
    
    -- 门店日清商品表
    is_clean                BIGINT COMMENT '商品是否日清',

    require_qty             DECIMAL(27, 3) COMMENT '要货数量',
    require_amount          DECIMAL(27, 2) COMMENT '要货金额'
)
COMMENT '门店商品要货刻表'
partitioned by(dt STRING COMMENT 'confirm_time确认时间')
row format delimited fields terminated by ','
stored as orc
tblproperties ('orc.compress'='SNAPPY');
处理说明:
需要对dwd_order_store_require_i表的数据进行过滤:
require_type_code=2  -- 选择要货类型为配送的
and is_canceled=0 -- 选择没有取消的
and collect_require_order_id is not null 
and collect_require_order_id <>'' -- 选择要货汇总单不为空的,即已经生效的要货单
数据插入:
-- 门店商品要货刻表
with t1 as  (
    select
        trade_date,
        week_trade_date,
        month_trade_date,
        hourly,
        quarter,
        quarters,
        store_no,
        store_name,
        store_sale_type,
        store_type_code,
        worker_num,
        store_area,
        city_id,
        city_name,
        region_code,
        region_name,
        is_day_clear,

        goods_no,

        sum(require_qty) as require_qty,
        sum(require_price * require_qty ) as require_amount

    from dwd.dwd_order_store_require_i
    where require_type_code = 2 and is_canceled = 0  and collect_require_order_id is not null  and collect_require_order_id != ''
    group by
        trade_date,
        week_trade_date,
        month_trade_date,
        hourly,
        quarter,
        quarters,
        store_no,
        store_name,
        store_sale_type,
        store_type_code,
        worker_num,
        store_area,
        city_id,
        city_name,
        region_code,
        region_name,
        is_day_clear,
        goods_no
)
insert overwrite table dwm.dwm_order_store_goods_require_quarter_i partition (dt)
select
    t1.trade_date,
    t1.week_trade_date,
    t1.month_trade_date,
    t1.hourly,
    t1.quarter,
    t1.quarters,
    t1.store_no,
    t1.store_name,
    t1.store_sale_type,
    t1.store_type_code,
    t1.worker_num,
    t1.store_area,
    t1.city_id,
    t1.city_name,
    t1.region_code,
    t1.region_name,
    t1.is_day_clear,
    t2.first_category_no,
    t2.first_category_name,
    t2.second_category_no,
    t2.second_category_name,
    t2.third_category_no,
    t2.third_category_name,
    t1.goods_no,
    t2.goods_name,
    t2.is_clear as  is_clean,
    t1.require_qty,
    t1.require_amount,
    t1.trade_date as dt
from t1 left join dim.dwd_dim_store_goods_i  t2
    on t1.store_no = t2.store_no and t1.goods_no = t2.goods_no and t2.dt = '2023-11-23'

你可能感兴趣的:(黑马甄选离线数仓项目,大数据,笔记,数据仓库)