接触flink不久,开始学习flink table相关信息,记录下来学习过程,供大家参考学习
一、引用的jar包
二、概念和通用API
1、创建表环境
TableEnvironment是通过调用静态TableEnvironment. getTableEnvironment()方法创建的,该方法带有一个StreamExecutionEnvironment或一个ExecutionEnvironment以及一个可选的TableConfig
// **********************
// FLINK STREAMING QUERY
// **********************
EnvironmentSettings fsSettings = EnvironmentSettings.newInstance().useOldPlanner().inStreamingMode().build();
StreamExecutionEnvironment fsEnv = StreamExecutionEnvironment.getExecutionEnvironment();
StreamTableEnvironment fsTableEnv = StreamTableEnvironment.create(fsEnv, fsSettings);
// or TableEnvironment fsTableEnv = TableEnvironment.create(fsSettings);
// ******************
// FLINK BATCH QUERY
// ******************
import org.apache.flink.api.java.ExecutionEnvironment;
import org.apache.flink.table.api.java.BatchTableEnvironment;
ExecutionEnvironment fbEnv = ExecutionEnvironment.getExecutionEnvironment();
BatchTableEnvironment fbTableEnv = BatchTableEnvironment.create(fbEnv);
2、在目录中注册表
TableableEnvironment维护按名称注册的表的目录。有两种类型的表,输入表和输出表。输入表可以在表API和SQL查询中引用,并提供输入数据。输出表可用于将Table API或SQL查询的结果发送到外部系统。
// get a TableEnvironment
TableEnvironment tableEnv = ...; // see "Create a TableEnvironment" section
// table is the result of a simple projection query
Table projTable = tableEnv.scan("X").select(...);
// register the Table projTable as table "projectedTable"
tableEnv.registerTable("projectedTable", projTable);
3、注册TableSource
TableSource提供对存储在存储系统中的外部数据的访问,如数据库(MySQL、HBase、…)、具有特定编码的文件(CSV、Apache [Parquet、Avro、ORC]、…)或消息传递系统(Apache Kafka、RabbitMQ、…)
// get a TableEnvironment
TableEnvironment tableEnv = ...; // see "Create a TableEnvironment" section
// create a TableSource
TableSource csvSource = new CsvTableSource("/path/to/file", ...);
// register the TableSource as table "CsvTable"
tableEnv.registerTableSource("CsvTable", csvSource);
4、注册TableSink
已注册TableSink可用于将Table API或SQL查询的结果发送到外部存储系统,例如数据库,键值存储,消息队列或文件系统(在不同的编码中,例如,CSV,Apache [Parquet] ,Avro,ORC],......)
// get a TableEnvironment
TableEnvironment tableEnv = ...; // see "Create a TableEnvironment" section
// create a TableSink
TableSink csvSink = new CsvTableSink("/path/to/file", ...);
// define the field names and types
String[] fieldNames = {"a", "b", "c"};
TypeInformation[] fieldTypes = {Types.INT, Types.STRING, Types.LONG};
// register the TableSink as table "CsvSinkTable"
tableEnv.registerTableSink("CsvSinkTable", fieldNames, fieldTypes, csvSink);