Testcontainers对GCP服务进行测试

在上一篇博客我介绍了如何用Testcontainers来做Java测试,并以Kafka为例。在我的项目中,还用到了GCP的一些服务,例如Bigquery,Testcontainers同样提供了对GCP服务的支持。因为官网对于Bigquery的模拟测试介绍的太简略了,我这里总结一下如何来模拟Bigquery。

官网介绍可以用以下代码来配置一个Bigquery container

BigQueryEmulatorContainer container = new BigQueryEmulatorContainer("ghcr.io/goccy/bigquery-emulator:0.4.3")

但是这样得到的container是无法新增一个dataset的,起码我没找到对应的方法。

我这里采用另一种方式来加载Bigquery

public GenericContainer bqSimulator = 
        new GenericContainer<>(
            DockerImageName.parse("ghcr.io/goccy/bigquery-emulator"))
            .withExposedPorts(9050,9060)
            .withCommand("--project=test-project", "--dataset=dataset1");

这种方式加载,就可指定创建一个名为test-project的项目,并创建一个dataset1

之后就可以按照正常的方式来调用Bigquery的功能了,例如创建table,写入数据,查询数据等。先获取Bigquery client

String url = String.format("http://%s:%d", bqSimulator.getHost(), bqSimulator.getMappedPort(9050));
BigQueryOptions options = BigQueryOptions
    .newBuilder()
    .setProjectId("test-project")
    .setHost(url)
    .setLocation(url)
    .setCredentials(NoCredentials.getInstance())
    .build();
BigQuery bigQuery = options.getService();

查看当前项目的dataset

Page datasets = bigQuery.listDatasets("test-project", DatasetListOption.pageSize(100));
if (datasets == null) {
    System.out.println("Dataset does not contain any models");
    return;
}    
datasets
.iterateAll()
.forEach(
    dataset -> System.out.printf("Success! Dataset ID: %s ", dataset.getDatasetId()));

建立一个table,写入测试数据,然后再查询

TableId tableId = TableId.of("dataset1", "abc");
Schema schema =
    Schema.of(
        Field.of("name", StandardSQLTypeName.STRING),
        Field.of("time", StandardSQLTypeName.TIMESTAMP));
TableDefinition tableDefinition = StandardTableDefinition.of(schema);
TableInfo tableInfo = TableInfo.newBuilder(tableId, tableDefinition).build();
bigQuery.create(tableInfo);

DatasetId datasetId = DatasetId.of("test-project", "dataset1");
Page tables = bigQuery.listTables(datasetId, TableListOption.pageSize(100));
tables.iterateAll().forEach(table -> System.out.print(table.getTableId().getTable() + "\n"));

QueryJobConfiguration jobconf = QueryJobConfiguration.of("insert into dataset1.abc (name, time) values ('test', '2023-01-01T00:00:00')");
bigQuery.query(jobconf);

String queryResult = "Select * from dataset1.abc;";
QueryJobConfiguration queryConfig =
        QueryJobConfiguration.newBuilder(queryResult).setUseLegacySql(true).build();
TableResult result = bigQuery.query(queryConfig);
result.iterateAll().forEach(rows -> rows.forEach(row -> System.out.println(row.getValue())));

以上就是对Bigquery进行模拟测试的方法。

你可能感兴趣的:(java)