flink cdc笔记（二）：flink cdc的两种开发方式

支持两种开发方式：SQL和DataStream的方式

1，sql的方式

 Mysql和oracle在cdc sql中的数据类型映射

MySQL CDC Connector — Flink CDC documentation

MySQL type	Flink SQL type
TINYINT	TINYINT
SMALLINT TINYINT UNSIGNED	SMALLINT
INT MEDIUMINT SMALLINT UNSIGNED	INT
BIGINT INT UNSIGNED	BIGINT
BIGINT UNSIGNED	DECIMAL(20, 0)
REAL FLOAT	FLOAT
DOUBLE	DOUBLE
NUMERIC(p, s) DECIMAL(p, s) where p <= 38	DECIMAL(p, s)
NUMERIC(p, s) DECIMAL(p, s) where 38 < p <= 65	STRING
BOOLEAN TINYINT(1) BIT(1)	BOOLEAN
DATE	DATE
TIME [(p)]	TIME [(p)]
TIMESTAMP [(p)] DATETIME [(p)]	TIMESTAMP [(p)]
CHAR(n)	CHAR(n)
VARCHAR(n)	VARCHAR(n)
BIT(n)	BINARY(⌈n/8⌉)

Oracle type	Flink SQL type
NUMBER(p, s <= 0), p - s < 3	TINYINT
NUMBER(p, s <= 0), p - s < 5	SMALLINT
NUMBER(p, s <= 0), p - s < 10	INT
NUMBER(p, s <= 0), p - s < 19	BIGINT
NUMBER(p, s <= 0), 19 <= p - s <= 38	DECIMAL(p - s, 0)
NUMBER(p, s > 0)	DECIMAL(p, s)
NUMBER(p, s <= 0), p - s > 38	STRING
FLOAT BINARY_FLOAT	FLOAT
DOUBLE PRECISION BINARY_DOUBLE	DOUBLE
NUMBER(1)	BOOLEAN
DATE TIMESTAMP [(p)]	TIMESTAMP [(p)] [WITHOUT TIMEZONE]
TIMESTAMP [(p)] WITH TIME ZONE	TIMESTAMP [(p)] WITH TIME ZONE
TIMESTAMP [(p)] WITH LOCAL TIME ZONE	TIMESTAMP_LTZ [(p)]
CHAR(n) NCHAR(n) NVARCHAR2(n) VARCHAR(n) VARCHAR2(n) CLOB NCLOB XMLType	STRING
BLOB ROWID	BYTES

SQL读出来的内容是的Tuple2<Boolean,Row>的格式

 

 

2，DataStream的方式

 DataStream读出来的内容可以设置成JSON的格式，方便解析

 {"before":null,"after":{"C_COLLECTORID":"W2","S_TIME":1655497161000,"S_NUM":{"scale":0,"value":"BA=="},"S_LASTESTTIME":1655916525000,"S_INITTIME":1655497161000,"W_WORKSHOP":null,"W_WORKCLASSID":null,"C_FACTORY":"2720"},"source":{"version":"1.5.4.Final","connector":"oracle","name":"oracle_logminer","ts_ms":1657262696008,"snapshot":"last","db":"ORCL.TEST","sequence":null,"schema":"FLINKUSER","table":"GD_WCOLLECTCURRENTSTAT","txId":null,"scn":"1135085","commit_scn":null,"lcr_position":null},"op":“c","ts_ms":1657262696008,"transaction":null}

 

 before就是数据改变前的内容，after就是数据改变后的内容，op为c代表新增的，下面是op的对应数据改变类型