消费kafka:
val createTableSql =
"""
|CREATE TABLE aggs_test(
| data VARCHAR,
| account_id VARCHAR,
| android_id VARCHAR,
| iccid VARCHAR,
| imei VARCHAR,
| imsi VARCHAR,
| operator VARCHAR,
| oa_id VARCHAR,
| pre_imei VARCHAR,
| serial VARCHAR,
| app_version VARCHAR,
| channel VARCHAR,
| install_server_time VARCHAR,
| uid VARCHAR,
| trusted_id VARCHAR,
| uid VARCHAR,
| utd_id VARCHAR,
| source_versioncode VARCHAR,
| wmac VARCHAR,
| idfa VARCHAR,
| key_chain_idfa VARCHAR,
| idfv VARCHAR,
| ip VARCHAR,
| message_type VARCHAR,
| ts AS TO_TIMESTAMP(FROM_UNIXTIME(server_timestamp, 'yyyy-MM-dd HH:mm:ss')),
| WATERMARK FOR ts AS ts - INTERVAL '1' SECOND --1s的watermark
|
|)
|WITH(
| 'connector.type' ='kafka',
| 'connector.version' = 'universal',
| 'connector.topic' = 'xxx',
| 'connector.startup-mode' = 'earliest-offset',
| 'connector.properties.0.key' = 'bootstrap.servers',
| 'connector.properties.0.value' = 'xxx',
| 'update-mode' = 'append',
| 'format.type' = 'json',
| 'format.derive-schema' = 'true'
|)
""".stripMargin```
SQL:
```sql
SELECT
ad_id,
data_date,
ad_location,
ad_platform,
TUMBLE_START(ts,INTERVAL '5' second) as window_start,
TUMBLE_END(ts,INTERVAL '5' second) as window_end,
COUNT(1) AS click_count
FROM aggs_temp_table
where event_type='adclick'
GROUP BY
ad_id,
data_date,
ad_location,
ad_platform,
TUMBLE(ts ,INTERVAL '5' second) -- 5s一个窗口