参照下文,下载编译源码
Calcite-[1]-Tutorial
运行CsvTest 总体感知csv样例工程
输出
Test ignored.
Fred
Eric
John
Wilma
Alice
100, Fred, 10, , , 30, 25, true, false, 1996-08-03
110, Eric, 20, M, San Francisco, 3, 80, null, false, 2001-01-01
110, John, 40, M, Vancouver, 2, null, false, true, 2002-05-03
120, Wilma, 20, F, , 1, 5, null, true, 2005-09-07
130, Alice, 40, F, Vancouver, 2, null, false, true, 2007-01-01
Fred
Eric
John
Wilma
Alice
Test ignored.
Sales
Marketing
Accounts
100, Fred, 10, , , 30, 25, true, false, 1996-08-03
110, Eric, 20, M, San Francisco, 3, 80, null, false, 2001-01-01
110, John, 40, M, Vancouver, 2, null, false, true, 2002-05-03
120, Wilma, 20, F, , 1, 5, null, true, 2005-09-07
130, Alice, 40, F, Vancouver, 2, null, false, true, 2007-01-01
100, Fred, 10, , , 30, 25, true, false, 1996-08-03
110, Eric, 20, M, San Francisco, 3, 80, null, false, 2001-01-01
110, John, 40, M, Vancouver, 2, null, false, true, 2002-05-03
120, Wilma, 20, F, , 1, 5, null, true, 2005-09-07
130, Alice, 40, F, Vancouver, 2, null, false, true, 2007-01-01
model.json
{
"version": "1.0",
"defaultSchema": "SALES",
"schemas": [
{
"name": "SALES",
"type": "custom",
"factory": "org.apache.calcite.adapter.csv.CsvSchemaFactory",
"operand": {
"directory": "sales"
}
}
]
}
CsvSchemaFactory主要扫描directory中的文件,每个文件构建一个table
(1)CsvSchemaFactory
CsvSchemaFactory根据directory构建CsvSchema
CsvSchemaFactory::create
根据json中的directory和flavor 构建一个CsvSchema(directoryFile, flavor);
简要如下:
public class CsvSchemaFactory implements SchemaFactory {
public static final CsvSchemaFactory INSTANCE = new CsvSchemaFactory();
private CsvSchemaFactory() {
}
public Schema create(SchemaPlus parentSchema, String name,
Map operand) {
final String directory = (String) operand.get("directory");
File directoryFile = new File(directory);
String flavorName = (String) operand.get("flavor");
CsvTable.Flavor flavor = CsvTable.Flavor.SCANNABLE;
return new CsvSchema(directoryFile, flavor);
}
}
(2)CsvSchema
CsvSchema:Schema mapped onto a directory of CSV files. Each table in the schema is a CSV file in that directory.
将文件和table做个映射,每个文件映射为一个table,从上面可以看出schema是虚拟的数据库,里面有tableMap,
可知道包含哪些表,Map
public class CsvSchema extends AbstractSchema {
private final File directoryFile;
private final CsvTable.Flavor flavor;
private Map tableMap;
CsvSchema::createTableMap
private Map createTableMap()
主要根据目录遍历所有 ".csv", ".csv.gz", ".json", ".json.gz",对遍历的文件,调用createTable构建表
CsvSchema::createTable
调用 CsvTranslatableTable or CsvScannableTable or CsvFilterableTable构建表
(3)CsvScannableTable
Table that can be scanned without creating an intermediate relational expression
class CsvScannableTable extends CsvTable implements ScannableTable
其中ScannableTable 接口:
Table that can be scanned without creating an intermediate relational expression
CsvScannableTable::scan 返回Enumerable
(4)debug查看执行过程
EMPS.csv.gz
解压后的文件内容为
EMPNO:int | NAME:string | DEPTNO:int | GENDER:string | CITY:string | EMPID:int | AGE:int | SLACKER:boolean | MANAGER:boolean | JOINEDAT:date |
100 | Fred | 10 | 30 | 25 | TRUE | FALSE | 1996/8/3 | ||
110 | Eric | 20 | M | San Francisco | 3 | 80 | FALSE | 2001/1/1 | |
110 | John | 40 | M | Vancouver | 2 | FALSE | TRUE | 2002/5/3 | |
120 | Wilma | 20 | F | 1 | 5 | TRUE | 2005/9/7 | ||
130 | Alice | 40 | F | Vancouver | 2 | FALSE | TRUE | 2007/1/1 |
@Test public void testSelectSingleProject() throws SQLException {
sql("smart", "select name from DEPTS").ok();
}
sql(xxx)调用创建Fluent
Fluent API to perform test actions.
Fluent::ok::checkSql(sql, model, expect)
checkSql中建立连接执行sql查询
connection = DriverManager.getConnection("jdbc:calcite:", info);
info中时mode的路径,这里会直接调用model.json中配置的CsvSchemaFactory
CsvSchemaFactory::create(xxx) 返回 Schema,从上面可以看出schema是虚拟的数据库,里面有tableMap,可知道包含哪些表,Map tableMap
CsvSchemaFactory::create(xxx) 返回一个CsvSchema
sql(xxx)调用创建Fluent
Fluent API to perform test actions.
Fluent::ok::checkSql(sql, model, expect)
checkSql中建立连接执行sql查询
connection = DriverManager.getConnection("jdbc:calcite:", info);
info中时mode的路径,这里会直接调用model.json中配置的CsvSchemaFactory
CsvSchemaFactory::create(xxx) 返回 Schema
(schema是虚拟的数据库,里面有tableMap,可知道包含哪些表,Map tableMap)
ModelHandler::visit(JsonCustomSchema jsonSchema) -->schemaFactory.create-->
connection.setSchema(jsonRoot.defaultSchema)-->connection.init()--返回 connection
ResultSet resultSet =connection.creatSatement().executeQuery(sql)
【注意】在执行sql时才开始调用CsvSchema::createTableMap 构建表,构建一个CsvScannableTable