MatrixOne从入门到实践——MO-Tester
从 0.5.0 版本开始,MatrixOne 引入了一个自动测试框架 MO-Tester。
MO-Tester 测试框架,也可以称作为测试器,是通过 SQL 测试 MatrixOne 或其他数据库功能的。
简介
MO-Tester 是基于 Java 语言进行开发,用于 MatrixOne 的测试套件。MO-Tester 构建了一整套完整的工具链来进行 SQL 自动测试。它包含测试用例和运行结果。MO-Tester 启动后,MO-Tester 将使用 MatrixOne 运行所有 SQL 测试用例,并将所有输出 SQL 测试结果与预期结果进行比较。所有案例的结果无论成功或者失败,都将记录在报告中。
MO-Tester 相关用例、结果和报告的链接如下:
- Cases: https://github.com/matrixorig...
- Result: https://github.com/matrixorig...
- Report: 运行结束后,本地目录自动生成
mo-tester/report
。
测试用例和测试结果一一对应。如需添加新的测试用例和测试结果请进入右侧所示 MatrixOne 仓库路径中进行添加:https://github.com/matrixorig...
MO-Tester 测试用例如下表所示:
测试用例 | 描述 |
---|---|
Benchmark/TPCH | DDL and 22 Queries of TPCH Benchmark |
Database | DDL Statements, creation/drop databases |
Table | DDL Statements, creation/drop tables |
DML | DML Statements, including insert, select, show statements |
dtype | Data Types and type conversion test cases |
Expression | Case when, With(CTE), Temporal Interval |
Function | Aggregate function, built-in function |
Explain | Explain statement |
Join | Join statement, including Left/Right/Inner/Outer/Natural Join |
Operator | Including +,-,*,/,MOD,%,=, >, <, IS, LIKE etc |
Subquery | Including Select/From/Where subquery |
Transaction | Test of isolation level, atomicity |
使用MO-Tester
环境准备
安装jdk 8
# 下载jdk压缩包 下载地址:http://www.oracle.com/technetwork/java/javase/downloads/index.html 下载版本为java 8 的: jdk-8u131-linux-x64.tar.gz # 解压 tar -zxf jdk-8u131-linux-x64.tar.gz # 解压后 的目录假设为 /home/java/jdk1.8.0_131 # 编辑/etc/profile,文件的末尾添加 export JAVA_HOME=/home/java/jdk1.8.0_131 export PATH=$PATH:$JAVA_HOME/bin # 刷新profile source /etc/profile # 验证 jdk [root@motest java]# java -version java version "1.8.0_131" Java(TM) SE Runtime Environment (build 1.8.0_131-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
启动 MatrixOne
请参考[MatrixOne部署]()
克隆 MO-Tester 仓库
git clone https://github.com/matrixorigin/mo-tester.git
配置 MO-Tester
MO-tester 基于 Java 语言进行开发,因此 Mo-tester 所依赖的 Java 数据库连接(JDBC,Java Database Connectivity) 驱动程序需要配置 mo.yml 文件里的参数信息:进入到 mo-tester 本地仓库,打开 mo.yml 文件,配置服务器地址、默认的数据库名称、用户名和密码等。
以下是本地独立版本 MatrixOne 的默认示例。
#jdbc
jdbc:
driver: "com.mysql.cj.jdbc.Driver"
server:
- addr: "127.0.0.1:6001"
#- addr: "127.0.0.1:3306"
#- addr: "127.0.0.1:3306"
database:
default: ""
paremeter:
characterSetResults: "utf8"
continueBatchOnError: "false"
useServerPrepStmts: "true"
alwaysSendSetIsolation: "false"
useLocalSessionState: "true"
zeroDateTimeBehavior: "CONVERT_TO_NULL"
failoverReadOnly: "false"
serverTimezone: "Asia/Shanghai"
socketTimeout: 30000
#users
user:
name: "dump"
password: "111"
运行MO-Tester
运行以下所示命令行,SQL 所有测试用例将自动运行,并将报告和错误消息生成至 report/report.txt 和 report/error.txt 文件中。
./run.sh
如果你想调整测试范围,你可以修改 run.yml
文件中的 path
参数。或者,在执行 run.sh
命令时,你也可以指定一些参数,参数解释如下:
参数 | 参数定义 |
---|---|
-p | 设置由 MO-tester 执行的测试用例的路径。默认值可以参见 run.yml 文件中 path 的配置参数 |
-m | 设置 MO-tester 测试的方法,即直接运行或者生成新的测试结果。默认值可以参见 run.yaml 文件中 method 的配置参数 |
-t | 设置 MO-tester 执行 SQL 命令的格式类型。默认值可以参见 run.yml 文件中 type 的配置参数。 |
-r | 设置测试用例应该达到的成功率。默认值可以参见 run.yml 文件中 rate 的配置参数。 |
-i | 设置包含列表,只有路径中名称包含其中一个列表的脚本文件将被执行,如果有多个,如果没有指定,用','分隔,指的是包含的所有情况set the including list, and only script files in the path whose name contains one of the lists will be executed, if more than one, separated by , , if not specified, refers to all cases included |
-e | 设置排除列表,如果路径下的脚本文件的名称包含一个排除列表,则不会被执行,如果有多个,用','分隔,如果没有指定,表示不排除任何情况set the excluding list, and script files in the path whose name contains one of the lists will not be executed, if more than one, separated by , , if not specified, refers to none of the cases excluded |
-g | 表示带有[-- @bvt:issue#{issueNO.}]标志的 SQL 命令将不会被执行,该标志以 [-- @bvt:issue#{issueNO.}]开始,以 [-- @bvt:issue]结束。例如, -- @bvt:issue#3236 select date_add("1997-12-31 23:59:59",INTERVAL "-10000:1" HOUR_MINUTE); select date_add("1997-12-31 23:59:59",INTERVAL "-100 1" YEAR_MONTH); -- @bvt:issue 这两个 SQL 命令与问题 #3236 相关联,它们将不会在 MO-tester 测试中执行,直到问题 #3236 修复后标签移除才可以在测试中执行。 |
-n | 表示在比较结果时将忽略结果集的元数据 |
run.yml:
#test scritps path
path: "cases/"
#method
#run: means execute test scripts configured by para[path] completely,and generate the test reprot,execution logs
#debug: means execute test scripts configured by para[path] completely, only print the result to console,do not generate the test reprot,execution logs
#genrs: means generate the expected result for the test scripts only
method: "run"
#rate: means the execution success rate,if the actual rate is less than this,the programe will exit with status 1
rate: 100
示例:
./run.sh -p case -m run -t script -r 100 -i select,subquery -e substring -g
使用MO-Tester完成TPCH
注意: 这里的tpch只是一个测试用例,仅用来测试数据库能够跑通tpch的22条查询语句,包含的数据条数较少。
获取MatrixOne源码
git clone https://github.com/matrixorigin/matrixone.git
复制test目录到MO-Tester
cp -r matrixone/test/cases/benchmark /home/motest/mo-tester/cases # 复制后,可以在MO-Tester中看到相关case 如下所示: root@motest cases]# pwd /home/motest/mo-tester/cases [root@motest cases]# cd benchmark/ [root@motest benchmark]# ll total 0 drwxr-xr-x. 6 root root 71 Oct 1 09:06 tpch [root@motest benchmark]# pwd /home/motest/mo-tester/cases/benchmark
同理,将MatrixOne源码目录中test下的result也复制过来
cp -r matrixone/test/result/benchmark /home/motest/mo-tester/result
运行MO-Tester进行TPCH测试
./run.sh -p ./cases/benchmark/tpch -m run -t script -r 100
最终结果显示:
The path of the cases that need to be executed by mo-tester : ./cases/benchmark/tpch The method that mo-tester will run with :run The type of the format that mo-tester execute the sqlcommand in : script The success rate that test cases should reach : 100 2022-10-01 09:18:23 INFO Tester:134 - The method is [run],now start to run the scripts in the path[./cases/benchmark/tpch]. 2022-10-01 09:18:23 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/01_DDL/01_create_table.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:23 INFO Executor:186 - The script file[./cases/benchmark/tpch/01_DDL/01_create_table.sql] has been executed, and cost: 0.02s, total:11, success:11, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:23 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/02_insert_customer.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:23 INFO Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/02_insert_customer.sql] has been executed, and cost: 0.032s, total:16, success:16, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:23 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/03_insert_lineitem.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:25 INFO Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/03_insert_lineitem.sql] has been executed, and cost: 1.584s, total:602, success:602, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:25 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/04_insert_nation.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:25 INFO Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/04_insert_nation.sql] has been executed, and cost: 0.008s, total:4, success:4, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:25 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/05_insert_orders.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:25 INFO Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/05_insert_orders.sql] has been executed, and cost: 0.288s, total:151, success:151, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:25 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/06_insert_part.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:25 INFO Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/06_insert_part.sql] has been executed, and cost: 0.031s, total:21, success:21, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:25 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/07_insert_partsupp.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/07_insert_partsupp.sql] has been executed, and cost: 0.114s, total:81, success:81, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/08_insert_region.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/08_insert_region.sql] has been executed, and cost: 0.004s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/09_insert_supplier.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/09_insert_supplier.sql] has been executed, and cost: 0.004s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/10_select_count.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/10_select_count.sql] has been executed, and cost: 0.016s, total:8, success:8, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q1.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q1.sql] has been executed, and cost: 0.028s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q10.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q10.sql] has been executed, and cost: 0.007s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q11.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q11.sql] has been executed, and cost: 0.005s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q12.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q12.sql] has been executed, and cost: 0.005s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q13.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q13.sql] has been executed, and cost: 0.006s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q14.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q14.sql] has been executed, and cost: 0.005s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q15.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q15.sql] has been executed, and cost: 0.004s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q16.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q16.sql] has been executed, and cost: 0.008s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q17.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q17.sql] has been executed, and cost: 0.005s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q18.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q18.sql] has been executed, and cost: 0.006s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q19.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q19.sql] has been executed, and cost: 0.01s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q2.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q2.sql] has been executed, and cost: 0.006s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q20.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q20.sql] has been executed, and cost: 0.005s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q21.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q21.sql] has been executed, and cost: 0.006s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q22.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q22.sql] has been executed, and cost: 0.007s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q3.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q3.sql] has been executed, and cost: 0.008s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q4.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q4.sql] has been executed, and cost: 0.005s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q5.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q5.sql] has been executed, and cost: 0.007s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q6.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q6.sql] has been executed, and cost: 0.006s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q7.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q7.sql] has been executed, and cost: 0.011s, total:3, success:3, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q8.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q8.sql] has been executed, and cost: 0.008s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q9.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q9.sql] has been executed, and cost: 0.011s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/04_CLEANUP/01_DROP.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/04_CLEANUP/01_DROP.sql] has been executed, and cost: 0.015s, total:9, success:9, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Tester:136 - All the scripts in the path[./cases/benchmark/tpch] have been excuted.Now start to create the test report. 2022-10-01 09:18:26 INFO TestReport:49 - [SUMMARY] COST : 1s, TOTAL :952, SUCCESS : 952, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/01_DDL/01_create_table.sql] COST : 0.02s, TOTAL :11, SUCCESS :11, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/02_LOAD/02_insert_customer.sql] COST : 0.032s, TOTAL :16, SUCCESS :16, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/02_LOAD/03_insert_lineitem.sql] COST : 1.584s, TOTAL :602, SUCCESS :602, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/02_LOAD/04_insert_nation.sql] COST : 0.008s, TOTAL :4, SUCCESS :4, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/02_LOAD/05_insert_orders.sql] COST : 0.288s, TOTAL :151, SUCCESS :151, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/02_LOAD/06_insert_part.sql] COST : 0.031s, TOTAL :21, SUCCESS :21, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/02_LOAD/07_insert_partsupp.sql] COST : 0.114s, TOTAL :81, SUCCESS :81, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/02_LOAD/08_insert_region.sql] COST : 0.004s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/02_LOAD/09_insert_supplier.sql] COST : 0.004s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/02_LOAD/10_select_count.sql] COST : 0.016s, TOTAL :8, SUCCESS :8, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q1.sql] COST : 0.028s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q10.sql] COST : 0.007s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q11.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q12.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q13.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q14.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q15.sql] COST : 0.004s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q16.sql] COST : 0.008s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q17.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q18.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q19.sql] COST : 0.01s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q2.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q20.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q21.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q22.sql] COST : 0.007s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q3.sql] COST : 0.008s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q4.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q5.sql] COST : 0.007s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q6.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q7.sql] COST : 0.011s, TOTAL :3, SUCCESS :3, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q8.sql] COST : 0.008s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q9.sql] COST : 0.011s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/04_CLEANUP/01_DROP.sql] COST : 0.015s, TOTAL :9, SUCCESS :9, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO Tester:138 - The test report has been generated in files[report.txt,report.xml]. 2022-10-01 09:18:26 INFO Tester:144 - The execution success rate is 100%, and not less than config value 100%,this test succeed.
我们可以看一下report.txt 观察测试用例结果
cd ./report/ [root@motest report]# cat report.txt [SUMMARY] COST : 1s, TOTAL :952, SUCCESS : 952, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/01_DDL/01_create_table.sql] COST : 0.02s, TOTAL :11, SUCCESS :11, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/02_LOAD/02_insert_customer.sql] COST : 0.032s, TOTAL :16, SUCCESS :16, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/02_LOAD/03_insert_lineitem.sql] COST : 1.584s, TOTAL :602, SUCCESS :602, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/02_LOAD/04_insert_nation.sql] COST : 0.008s, TOTAL :4, SUCCESS :4, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/02_LOAD/05_insert_orders.sql] COST : 0.288s, TOTAL :151, SUCCESS :151, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/02_LOAD/06_insert_part.sql] COST : 0.031s, TOTAL :21, SUCCESS :21, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/02_LOAD/07_insert_partsupp.sql] COST : 0.114s, TOTAL :81, SUCCESS :81, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/02_LOAD/08_insert_region.sql] COST : 0.004s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/02_LOAD/09_insert_supplier.sql] COST : 0.004s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/02_LOAD/10_select_count.sql] COST : 0.016s, TOTAL :8, SUCCESS :8, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q1.sql] COST : 0.028s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q10.sql] COST : 0.007s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q11.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q12.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q13.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q14.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q15.sql] COST : 0.004s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q16.sql] COST : 0.008s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q17.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q18.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q19.sql] COST : 0.01s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q2.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q20.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q21.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q22.sql] COST : 0.007s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q3.sql] COST : 0.008s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q4.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q5.sql] COST : 0.007s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q6.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q7.sql] COST : 0.011s, TOTAL :3, SUCCESS :3, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q8.sql] COST : 0.008s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q9.sql] COST : 0.011s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/04_CLEANUP/01_DROP.sql] COST : 0.015s, TOTAL :9, SUCCESS :9, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
同理,我们如果需要执行其他的测试用例,只需要将对应的cases复制过来,运行时指定对应的路径即可