Calcite-[5]-example-csv简析-CsvScannableTable

一 源码下载

参照下文，下载编译源码

Calcite-[1]-Tutorial

运行CsvTest 总体感知csv样例工程

输出

Test ignored.
Fred
Eric
John
Wilma
Alice
100, Fred, 10, , , 30, 25, true, false, 1996-08-03
110, Eric, 20, M, San Francisco, 3, 80, null, false, 2001-01-01
110, John, 40, M, Vancouver, 2, null, false, true, 2002-05-03
120, Wilma, 20, F, , 1, 5, null, true, 2005-09-07
130, Alice, 40, F, Vancouver, 2, null, false, true, 2007-01-01
Fred
Eric
John
Wilma
Alice

Test ignored.
Sales
Marketing
Accounts

100, Fred, 10, , , 30, 25, true, false, 1996-08-03
110, Eric, 20, M, San Francisco, 3, 80, null, false, 2001-01-01
110, John, 40, M, Vancouver, 2, null, false, true, 2002-05-03
120, Wilma, 20, F, , 1, 5, null, true, 2005-09-07
130, Alice, 40, F, Vancouver, 2, null, false, true, 2007-01-01
100, Fred, 10, , , 30, 25, true, false, 1996-08-03
110, Eric, 20, M, San Francisco, 3, 80, null, false, 2001-01-01
110, John, 40, M, Vancouver, 2, null, false, true, 2002-05-03
120, Wilma, 20, F, , 1, 5, null, true, 2005-09-07
130, Alice, 40, F, Vancouver, 2, null, false, true, 2007-01-01

二、简析1-model.json

model.json

{
  "version": "1.0",
  "defaultSchema": "SALES",
  "schemas": [
    {
      "name": "SALES",
      "type": "custom",
      "factory": "org.apache.calcite.adapter.csv.CsvSchemaFactory",
      "operand": {
        "directory": "sales"
 }
    }
  ]
}

CsvSchemaFactory主要扫描directory中的文件，每个文件构建一个table

（1）CsvSchemaFactory
CsvSchemaFactory根据directory构建CsvSchema
CsvSchemaFactory::create
根据json中的directory和flavor 构建一个CsvSchema(directoryFile, flavor);

简要如下：

public class CsvSchemaFactory implements SchemaFactory {
 public static final CsvSchemaFactory INSTANCE = new CsvSchemaFactory();
  private CsvSchemaFactory() {
  }
  public Schema create(SchemaPlus parentSchema, String name,
      Map operand) {
    final String directory = (String) operand.get("directory");
    File directoryFile = new File(directory);
    String flavorName = (String) operand.get("flavor");
    CsvTable.Flavor flavor = CsvTable.Flavor.SCANNABLE;
    return new CsvSchema(directoryFile, flavor);
  }
}

（2）CsvSchema

CsvSchema：Schema mapped onto a directory of CSV files. Each table in the schema is a CSV file in that directory.
将文件和table做个映射，每个文件映射为一个table，从上面可以看出schema是虚拟的数据库，里面有tableMap，
可知道包含哪些表，Map tableMap

public class CsvSchema extends AbstractSchema {
  private final File directoryFile;
  private final CsvTable.Flavor flavor;
  private Map tableMap;

CsvSchema::createTableMap
 private Map createTableMap()
        主要根据目录遍历所有 ".csv", ".csv.gz", ".json", ".json.gz"，对遍历的文件，调用createTable构建表

CsvSchema::createTable
   调用 CsvTranslatableTable or CsvScannableTable or CsvFilterableTable构建表

(3)CsvScannableTable

Table that can be scanned without creating an intermediate relational expression

class CsvScannableTable extends CsvTable implements ScannableTable
   其中ScannableTable 接口：
        Table that can be scanned without creating an intermediate relational expression
   CsvScannableTable::scan 返回Enumerable
        主要方法scan,调用CsvEnumerator   
        Enumerator that reads from a CSV file.
abstract class CsvTable extends AbstractTable
其中，abstract class AbstractTable implements Table, Wrapper
    final Source source;
    final RelProtoDataType protoRowType;
    List fieldTypes;
     
    CsvTable::CsvFieldType成员
    是个枚举，主要是列举那些可用的类型

（4）debug查看执行过程

EMPS.csv.gz

解压后的文件内容为

EMPNO:int	NAME:string	DEPTNO:int	GENDER:string	CITY:string	EMPID:int	AGE:int	SLACKER:boolean	MANAGER:boolean	JOINEDAT:date
100	Fred	10			30	25	TRUE	FALSE	1996/8/3
110	Eric	20	M	San Francisco	3	80		FALSE	2001/1/1
110	John	40	M	Vancouver	2		FALSE	TRUE	2002/5/3
120	Wilma	20	F		1	5		TRUE	2005/9/7
130	Alice	40	F	Vancouver	2		FALSE	TRUE	2007/1/1

@Test public void testSelectSingleProject() throws SQLException {
  sql("smart", "select name from DEPTS").ok();
}
sql(xxx)调用创建Fluent

 Fluent API to perform test actions.
   Fluent::ok::checkSql(sql, model, expect)
checkSql中建立连接执行sql查询
     connection = DriverManager.getConnection("jdbc:calcite:", info);
          info中时mode的路径，这里会直接调用model.json中配置的CsvSchemaFactory

     CsvSchemaFactory::create(xxx) 返回 Schema，从上面可以看出schema是虚拟的数据库，里面有tableMap，可知道包含哪些表，Map tableMap
     CsvSchemaFactory::create(xxx) 返回一个CsvSchema

sql(xxx)调用创建Fluent
 Fluent API to perform test actions.
   Fluent::ok::checkSql(sql, model, expect)
checkSql中建立连接执行sql查询
     connection = DriverManager.getConnection("jdbc:calcite:", info);
          	info中时mode的路径，这里会直接调用model.json中配置的CsvSchemaFactory
     		CsvSchemaFactory::create(xxx) 返回 Schema
     		（schema是虚拟的数据库，里面有tableMap，可知道包含哪些表，Map tableMap）
     		ModelHandler::visit(JsonCustomSchema jsonSchema) -->schemaFactory.create-->
     		connection.setSchema(jsonRoot.defaultSchema)-->connection.init()--返回 connection
     ResultSet resultSet =connection.creatSatement().executeQuery(sql)
            【注意】在执行sql时才开始调用CsvSchema::createTableMap  构建表，构建一个CsvScannableTable