coverage代码覆盖率测试介绍

coverage代码覆盖率测试介绍

背景知识补充

1、什么是覆盖率

测试过程中提到的覆盖率，指的是已测试的内容，占待测内容的百分比，在一定程度上反应测试的完整程度。

覆盖率有可以根据要衡量的对象细分很多种，比如接口覆盖率、分支覆盖率、行覆盖率等等

2、做覆盖率有什么用处

覆盖率的好处是可以将测试的完整性量化，可以作为补充测试的手段，也可以在一定程度上佐证测试结果的可靠性。

---

简介

代码覆盖率测量通常用于衡量测试的有效性。它可以显示你的代码的哪些部分正在被测试执行，哪些没有被执行。

coverage是一个测量 Python 程序代码覆盖率的工具。它监视你的程序，并分析源码生成代码覆盖率报告。

---

安装

coverage依赖python环境

python环境安装：https://computingforgeeks.com/install-latest-python-on-centos-linux/

安装coverage

python3 -m pip install coverage
or
pip3 install coverage

安装完成

coverage --version
Coverage.py, version 7.2.5 with C extension
Full documentation is at https://coverage.readthedocs.io/en/7.2.5

---

命令概览

coverage支持命令如下

[yhgao@localhost ~]$ coverage --help
Coverage.py, version 7.2.5 with C extension
Measure, collect, and report on code coverage in Python programs.

usage: coverage  [options] [args]

Commands:
    annotate    Annotate source files with execution information.
    combine     Combine a number of data files.
    debug       Display information about the internals of coverage.py
    erase       Erase previously collected coverage data.
    help        Get help on using coverage.py.
    html        Create an HTML report.
    json        Create a JSON report of coverage results.
    lcov        Create an LCOV report of coverage results.
    report      Report coverage stats on modules.
    run         Run a Python program and measure code execution.
    xml         Create an XML report of coverage results.

Use "coverage help " for detailed help on any command.
Full documentation is at https://coverage.readthedocs.io/en/7.2.5

其中每个小命令后面可以查看更详细的帮助， 以run命令举例

[yhgao@localhost ~]$ coverage run --help
Usage: coverage run [options]  [program options]

Run a Python program, measuring code execution.

Options:
  -a, --append          Append coverage data to .coverage, otherwise it starts
                        clean each time.
  --branch              Measure branch coverage in addition to statement
                        coverage.
  --concurrency=LIBS    Properly measure code using a concurrency library.
                        Valid values are: eventlet, gevent, greenlet,
                        multiprocessing, thread, or a comma-list of them.
  --context=LABEL       The context label to record for this coverage run.
  --data-file=OUTFILE   Write the recorded coverage data to this file.
                        Defaults to '.coverage'. [env: COVERAGE_FILE]
  --include=PAT1,PAT2,...
                        Include only files whose paths match one of these
                        patterns. Accepts shell-style wildcards, which must be
                        quoted.
  -m, --module           is an importable Python module, not a script
                        path, to be run as 'python -m' would run it.
  --omit=PAT1,PAT2,...  Omit files whose paths match one of these patterns.
                        Accepts shell-style wildcards, which must be quoted.
  -L, --pylib           Measure coverage even inside the Python installed
                        library, which isn't done by default.
  -p, --parallel-mode   Append the machine name, process id and random number
                        to the data file name to simplify collecting data from
                        many processes.
  --source=SRC1,SRC2,...
                        A list of directories or importable names of code to
                        measure.
  --timid               Use a simpler but slower trace method. Try this if you
                        get seemingly impossible results!
  --debug=OPTS          Debug options, separated by commas. [env:
                        COVERAGE_DEBUG]
  -h, --help            Get help on this command.
  --rcfile=RCFILE       Specify configuration file. By default '.coveragerc',
                        'setup.cfg', 'tox.ini', and 'pyproject.toml' are
                        tried. [env: COVERAGE_RCFILE]

Full documentation is at https://coverage.readthedocs.io/en/7.2.5

---

快速使用

如果你的测试运行程序命令以“python”开头，只需将初始的“python”替换为“coverage run”即可。

python something.py -->> becomes -->> coverage run something.py

python -m amodule -->> becomes -->> coverage run -m amodule

python其他单元测试框架

pytest
pytest arg1 arg2 arg3 -->> becomes -->> coverage run -m pytest arg1 arg2 arg3

unittest
python -m unittest discover -->> becomes -->>  coverage run -m unittest discover

nosetest
nosetests arg1 arg2 -->> becomes -->> coverage run -m nose arg1 arg2

测试完成后会在当前目录下默认生成一个.coverage文件，这里面保存了代码覆盖率结果，也可以通过–data-file生成指定名称的文件

直接查看

coverage report

$ coverage report -m
Name                      Stmts   Miss  Cover   Missing
-------------------------------------------------------
my_program.py                20      4    80%   33-35, 39
my_other_module.py           56      6    89%   17-23
-------------------------------------------------------
TOTAL                        76     10    87%

其他查看方式

使用其他格式查看，例如html，会生成html文件，使用浏览器打开index.html即可查看

coverage html

[yhgao@localhost coverage]$ coverage html
Wrote HTML report to htmlcov/index.html
[yhgao@localhost coverage]$ ll
总用量 4
-rw-rw-r-- 1 yhgao yhgao 679 6月  15 16:49 demo.py
drwxrwxr-x 2 yhgao yhgao 190 6月  15 16:51 htmlcov

[yhgao@localhost coverage]$ cd htmlcov/
[yhgao@localhost htmlcov]$ ls -al
总用量 92
drwxrwxr-x 2 yhgao yhgao   190 6月  15 16:51 .
drwxrwxr-x 3 yhgao yhgao    53 6月  15 16:51 ..
-rw-rw-r-- 1 yhgao yhgao 21359 6月  15 16:51 coverage_html.js
-rw-rw-r-- 1 yhgao yhgao 11790 6月  15 16:51 demo_py.html
-rw-rw-r-- 1 yhgao yhgao  1732 6月  15 16:51 favicon_32.png
-rw-rw-r-- 1 yhgao yhgao    27 6月  15 16:51 .gitignore
-rw-rw-r-- 1 yhgao yhgao  3816 6月  15 16:51 index.html  //浏览器打开该文件
-rw-rw-r-- 1 yhgao yhgao  9004 6月  15 16:51 keybd_closed.png
-rw-rw-r-- 1 yhgao yhgao  9003 6月  15 16:51 keybd_open.png
-rw-rw-r-- 1 yhgao yhgao   236 6月  15 16:51 status.json
-rw-rw-r-- 1 yhgao yhgao 12387 6月  15 16:51 style.css

---

Demo

下面以一个demo.py举例

测试目标（%）：100

测试case数量（个）：3

//查看demo文件
[yhgao@bogon coverage]$ cat demo.py
import sys

num = int(sys.argv[1])
if num == 1:
        print('output num: 1')
else:
        if num == 2:
                print('output num: 2')
        elif (num > 2 or num < 1):
                print('error input num')
                
//python执行结果
[yhgao@bogon coverage]$ python demo.py 1
output num: 1
[yhgao@bogon coverage]$ python demo.py 2
output num: 2
[yhgao@bogon coverage]$ python demo.py 3
error input num

//coverage执行代码覆盖率（区分单个case情况）
[yhgao@bogon coverage]$ coverage run --data-file=c1 demo.py 1
output num: 1
[yhgao@bogon coverage]$ coverage run --data-file=c2 demo.py 2
output num: 2
[yhgao@bogon coverage]$ coverage run --data-file=c3 demo.py 3
error input num

//查看每个代码覆盖率文件
[yhgao@bogon coverage]$ coverage report --data-file=c1
Name      Stmts   Miss  Cover
-----------------------------
demo.py       8      4    50%
-----------------------------
TOTAL         8      4    50%
[yhgao@bogon coverage]$ coverage report --data-file=c2
Name      Stmts   Miss  Cover
-----------------------------
demo.py       8      3    62%
-----------------------------
TOTAL         8      3    62%
[yhgao@bogon coverage]$ coverage report --data-file=c3
Name      Stmts   Miss  Cover
-----------------------------
demo.py       8      2    75%
-----------------------------
TOTAL         8      2    75%

//合并代码覆盖率文件
[yhgao@bogon coverage]$ coverage combine --data-file=cc c1 c2 c3
Combined data file c1
Combined data file c2
Combined data file c3

//查看合并后的代码覆盖率文件
[yhgao@bogon coverage]$ coverage report --data-file=cc
Name      Stmts   Miss  Cover
-----------------------------
demo.py       8      0   100%
-----------------------------
TOTAL         8      0   100%

可以看到整合之后的代码覆盖率达到100%，我们也就可以据此判断我们的3个测试case是符合要求的。

//coverage执行代码覆盖率（不区分单个case情况）
未指定-a的情况：
[yhgao@bogon coverage]$ coverage run --data-file=cc2 demo.py 1
output num: 1
[yhgao@bogon coverage]$ coverage run --data-file=cc2 demo.py 2
output num: 2
[yhgao@bogon coverage]$ coverage run --data-file=cc2 demo.py 3
error input num
[yhgao@bogon coverage]$ coverage report --data-file=cc2
Name      Stmts   Miss  Cover
-----------------------------
demo.py       8      2    75%
-----------------------------
TOTAL         8      2    75%

代码覆盖率是会被覆盖的，所以只保留最后一个case的代码覆盖率

指定-a的情况：
[yhgao@bogon coverage]$ coverage run -a --data-file=cc2 demo.py 1
output num: 1
[yhgao@bogon coverage]$ coverage run -a --data-file=cc2 demo.py 2
output num: 2
[yhgao@bogon coverage]$ coverage run -a --data-file=cc2 demo.py 3
error input num
[yhgao@bogon coverage]$ coverage report --data-file=cc2
Name      Stmts   Miss  Cover
-----------------------------
demo.py       8      0   100%
-----------------------------
TOTAL         8      0   100%

指定-a和同一个data-file即可不用执行coverage combine，直接得到一份儿完整的代码覆盖率文件