VK-GL-CTS是Khrone的CTS测试代码
配置
下载
github上下载代码：GitHub - KhronosGroup/VK-GL-CTS: Khronos Vulkan, OpenGL, and OpenGL ES Conformance Tests
编译方法参考代码目录下的根目录下的 REASDME.md
可以跑vulkan和opengl，这里只跑opengl，参见：external/openglcts/README.md
源码下载完后需要下载依赖：zlib, libpng, glslang, spirv-headers, spirv-tools，renderdoc
python external/fetch_sources.py
这个下载需要登录github的账号和密码，github网页上登陆后，可以设置ssh-key。如果下载失败，也可以手动下载放到源码external下的对应目录，下载时注意需要的版本，试了几遍，我的一直是zlib、libpng和renderdoc下载失败，这三个手动下载放到对应地方，fetch脚本中注释掉这三个，后面的顺利下载：
image.png
编译
vulkan的编译比较简单，在cmake目录按照cmake常规选项进行cmake后make即可。
opengl的编译需要指定依赖平台和API类型版本
mkdir build;
cd build;
cmake -DDEQP_TARGET=null -DGLCTS_GTF_TARGET=gles32
cmake --build external/openglcts
DEQP_TARGET的参数：
| Name | Description |
|---|---|
| android | Used in Android build. Requires use of suitable toolchain file (see cmake/ directory) |
| default | Checks for presence of GL, ES2, ES3, and EGL libraries and headers in default search paths and configures build accordingly |
| null | Null build target |
| nullws | NullWS build target |
| x11_egl | X11 build for platforms with native EGL support |
| x11_glx | X11 build for platforms with native GLX support |
| x11_egl_glx | X11 build for platforms with native EGL/GLX support |
GLCTS_GTF_TARGET：gles2, gles3, gles31, gles32, and gl. default：gles32.
运行和结果分析：
运行
进入buld/external/openglcts/modules/目录下
./cts-runner.exe --type=glxy
[x and y are the major and minor specifiction versions]
Full list of parameters for the cts-runner binary:
--type=[esN[M]|glNM] Conformance test run type. Choose from
ES: es2, es3, es31, es32
GL: gl30, gl31, gl32, gl33, gl40, gl41, gl42, gl43, gl44, gl45, gl46
--waivers=[path] Path to xml file containing waived tests
--logdir=[path] Destination directory for log files
--summary Print summary without running the tests
--verbose Print out and log more information
结果文件：qpa和xml的一个总结文件，默认是不打开log和info信息输出的。
部分运行：
Debug/glcts.exe --deqp-runmode=txt-caselist
列出单跑的testcase列表，如下运行：
Debug/glcts.exe --deqp-case=KHR-GLES3.shaders.loops.* --deqp-gl-config-id=3
其他
-h, --help
-n, --deqp-case=
Test case(s) to run, supports wildcards (e.g. dEQP-GLES2.info.*)
--deqp-caselist=
Case list to run in trie format (e.g. {dEQP-GLES2{info{version,renderer}}})
--deqp-caselist-file=
Read case list (in trie format) from given file
--deqp-caselist-resource=
Read case list (in trie format) from given file located application's assets
--deqp-stdin-caselist
Read case list (in trie format) from stdin
--deqp-log-filename=
Write test results to given file
default: 'TestResults.qpa'
--deqp-runmode=[execute|xml-caselist|txt-caselist|stdout-caselist]
Execute tests, or write list of test cases into a file
default: 'execute'
--deqp-caselist-export-file=
Set the target file name pattern for caselist export
default: '${packageName}-cases.${typeExtension}'
--deqp-watchdog=[enable|disable]
Enable test watchdog
default: 'disable'
--deqp-crashhandler=[enable|disable]
Enable crash handling
default: 'disable'
--deqp-base-seed=
Base seed for test cases that use randomization
default: '0'
--deqp-test-iteration-count=
Iteration count for cases that support variable number of iterations
default: '0'
--deqp-visibility=[windowed|fullscreen|hidden]
Default test window visibility
default: 'windowed'
--deqp-surface-width=
Use given surface width if possible
default: '-1'
--deqp-surface-height=
Use given surface height if possible
default: '-1'
--deqp-surface-type=[window|pixmap|pbuffer|fbo]
Use given surface type
default: 'window'
--deqp-screen-rotation=[unspecified|0|90|180|270]
Screen rotation for platforms that support it
default: '0'
--deqp-gl-context-type=
OpenGL context type for platforms that support multiple
--deqp-gl-config-id=
OpenGL (ES) render config ID (EGL config id on EGL platforms)
default: '-1'
--deqp-gl-config-name=
Symbolic OpenGL (ES) render config name
--deqp-gl-context-flags=
OpenGL context flags (comma-separated, supports debug and robust)
--deqp-cl-platform-id=
Execute tests on given OpenCL platform (IDs start from 1)
default: '1'
--deqp-cl-device-ids=
Execute tests on given CL devices (comma-separated, IDs start from 1)
default: ''
--deqp-cl-build-options=
Extra build options for OpenCL compiler
--deqp-egl-display-type=
EGL native display type
--deqp-egl-window-type=
EGL native window type
--deqp-egl-pixmap-type=
EGL native pixmap type
--deqp-log-images=[enable|disable]
Enable or disable logging of result images
default: 'enable'
--deqp-log-shader-sources=[enable|disable]
Enable or disable logging of shader sources
default: 'enable'
--deqp-test-oom=[enable|disable]
Run tests that exhaust memory on purpose
default: 'enable'
--deqp-archive-dir=
Path to test resource files
default: '.'
--deqp-log-flush=[enable|disable]
Enable or disable log file fflush
default: 'enable'
--deqp-renderdoc=[enable|disable]
Enable RenderDoc frame markers
default: 'disable'
--deqp-fraction=
Run a fraction of the test cases (e.g. N,M means run group%M==N)
default: ''
--deqp-fraction-mandatory-caselist-file=
Case list file that must be run for each fraction
default: ''
--deqp-waiver-file=
Read waived tests from given file
default: ''
--deqp-runner-type=[any|none|amber]
Filter test cases based on runner
default: 'any'
--deqp-terminate-on-fail=[enable|disable]
Terminate the run on first failure
default: 'disable'
--deqp-egl-config-id=
Legacy name for --deqp-gl-config-id
default: '-1'
--deqp-egl-config-name=
Legacy name for --deqp-gl-config-name
--deqp-waiver-file=
Path to xml file containing waived tests
结果查看
汇总文件cts-run-summary.xml中包含了每一个testcase的结果路径，每个testcase的结果存在qpa文件中。还提供工具对结果进行验证，格式转换等，可以通过http来打开查看，也有工具。
image.png
To analyse and process the log files, run the following scripts
-
verify_submission.pylocated in VK-GL-CTS-Tools: Script that verifies logs based oncts-run-summary.xmlfile. -
scripts/log/log_to_csv.py: This utility converts.qpalog into CSV format. This is
useful for importing results into other systems. -
scripts/log/log_to_xml.py: Converts.qpainto well-formed XML document. The document
can be then viewed in browser using the testlog.{xsl,css} files.
Parser for the .qpa log file format in python is provided in
scripts/log/log_parser.py.