2019独角兽企业重金招聘Python工程师标准>>>
最新教程
自动化测试框架pytest教程
项目简介
这里是python测试开发的日常记录,主要涉及单元测试、安全测试、python基础、性能测试等内容。
项目地址:https://bitbucket.org/xurongzhong/small_python_daily_tools
项目介绍
从oracle迁移数据到mysql
-
代码位置: db/oracle 文件:config.py transfer_db.py
-
测试环境:CentOS release 6.6 (Final) 64 python2.7
-
运行依赖: python外部库cx_Oracle、mysql-connector-python
-
功能:迁移oracle数据到mysql,需要先行在mysql创建表。特别注意,有清空mysql表的动作。
python批量清空数据库
-
简介:使用python批量清空数据库,基于mysql python api和python标准库configparser,实现可配置地批量清空mysql表。
-
支持版本:python2.7/3.*
-
后期规划:通过argparse支持指定配置文件,增加delete、truncate、rename等选项。
-
代码 db/mysql del_db.py db.conf
在oracle中插入假数据
-
简介:在oracle中插入假数据
-
测试环境:CentOS release 6.6 (Final) 64 python2.7
-
代码 db/oracle oracle_add.py
简介
pytest是强大的python单元测试框架。由于python的胶水语言特性,实质上pytest在接口测试、自动化测试等方面也广泛使用。
-
成熟全功能的Python测试工具
- 支持POSIX / Windows, Python的2.6-3.5, PyPy和Jython - 2.5.1(也可能支持更高版本)
- 免费、开源,基于MIT license
- 1000测试用例自测。
- pytest升级时有很好的向后兼容性
- 丰富的在线和PDF文档 comprehensive onlinePDF documentation
- 大量的第三方插件和内置帮助builtin helpers, third party plugins
- 在许多小型和大型项目和组织使用,many small and large projects and organisations
- 许多测试实例,tested examples
-
灵活
- 易学,有多种用法
- assert语句断言
- traceback 和失败断言报告
- 打印调试和测试执行期间可以捕获标准输出
-
适合简单的单元测试到复杂的功能测试
- 模块化和参数化的平台
- 参数化的测试函数
- 支持属性
- Skip和xfail处理失败的用例
- xdist插件分发测试给多个CPU
- 不断地重新运行失败的测试
- 灵活的Python测试发现
-
集成
- 多范式:可以执行nose, unittest 和doctest风格的测试用例,甚至Django和trial。
- 支持良好的集成实践
- 支持扩展的xUnit风格setup
- 支持非python测试
- 支持生成测试覆盖率报告
- 支持PEP8兼容的编码风格
-
扩展插件和定制系统:
- 所有的收集,报告,运行方面都委派给hook函数
- 定制可以是每个目录,每个项目或每个PyPI上发布的插件
- 很容易添加命令行选项或定制现有的行为
- 很容易自己写插件
本节来源:http://pytest.org/latest/
交流:python开发自动化测试群291184506 PythonJava单元白盒单元测试群144081101 商务合作微信:pythontesting
英文原版书籍下载:https://bitbucket.org/xurongzhong/python-chinese-library/downloadss。
精品文章推荐:
python 2.7 中文教程及自动化测试介绍
使用Python学习selenium测试工具
性能测试艺术
Java单元测试之模拟利器-使用PowerMock进行Mock测试
入门基础
使用和调用
-
python -m pytest调用:
- python -m pytest [...] 效果和py.test [...] 一样
-
获取版本,选项名,环境变量
- py.test --version 看版本
- py.test --fixtures 查看内置函数参数
- py.test -h | --help 命令行和配置文件帮助
-
失败后停止执行
- 首次失败后停止执行:py.test -x
- py.test --maxfail=2 两次失败之后停止执行
-
执行选择用例
- py.test test_mod.py,执行模块中的用例
- py.test somepath,执行路径中用例
- py.test -k stringexpr,执行字符串表达式中的用例,比如"MyClass and not method"",选择TestMyClass.test_something,排除了TestMyClass.test_method_simple。
- py.test --pyargs pkg,导入pkg,使用其文件系统位置来查找和执行用例。执行pypkg目录下的所有用例。
- py.test test_mod.py::test_func 指定模块和函数
- py.test test_mod.py::TestClass::test_method 指定模块和类及方法
-
调试输出:
- py.test --showlocals 在traceback中显示本地变量
- py.test -l 在traceback中显示本地变量(快捷方式)
- py.test --tb=auto 默认格式,首尾为long,其他为short
- py.test --tb=long 详细的traceback信息格式化形式
- py.test --tb=native 标准库格式化形式
- py.test --tb=short 更短的traceback格式
- py.test --tb=line 每个错误一行
- py.test --tb=no 无traceback
- py.test --full-trace 最详细的格式
-
失败时调用PDB (Python Debugger):
Python带有一个内置的Python调试器称为PDB。pytest可以在命令行选项指定调用:
py.test --pdb
这将每次失败时调用Python调试器。
py.test -x --pdb # 失败时调用pdb,然后退出测试。 py.test --pdb - maxfail=3# 前3次失败调用pdb。
异常信息保存在sys.last_value
, sys.last_type和``sys.last_traceback,交互式情况可以调试:
>>> import sys
>>> sys.last_traceback.tb_lineno
42
>>> sys.last_value
AssertionError('assert result == "ok"',)
- 设置断点:
import pytest
def test_function():
...
pytest.set_trace() # invoke PDB debugger and tracing
2.0.0以前的版本中只有通过py.test-s禁用命令行捕捉才可以进入pdb调试。2.0.0及以后版本在进入pdb调试时自动禁用输出捕捉。
2.4.0 支持使用importpdb;pdb.set_trace()进入pdb。
- Profiling测试执行时间:得到最执行慢的10个测试:
py.test --durations=10
- 创建JUnitXML格式的文件
创建Jenkins或其他持续集成服务器的结果文件:
py.test --junitxml=path
- record_xml_property
2.8新增
在xml文件中记录额外信息
def test_function(record_xml_property):
record_xml_property("example_key", 1)
assert 0
效果如下:
试验功能,与pytest-xdist不兼容,还有可能破坏模式验证,与一些CI集成可能会有问题。
- 创建resultlog格式的文件
要创建纯文本的机器可读的结果文件,用于PyPy-testweb展示等。
py.test --resultlog=path
- 发送测试报告给在线pastebin服务
bpaste可以为你的文本生成url连接,下面为创建每个测试失败创建一个url:
py.test --pastebin=failed
py.test --pastebin=all py.test --pastebin=failed -x # 只发送一次
目前只支持:http://bpaste.net
-
禁用插件
py.test -p no:doctest
-
在python代码中调用pytest
2.0新增 用exitcode代替了SystemExit。 pytest.main([’-x’, ’mytestdir’]) pytest.main("-x mytestdir")
# 指定插件
# content of myinvoke.py import pytest class MyPlugin: def pytest_sessionfinish(self): print("*** test run reporting finishing") pytest.main("-qq", plugins=[MyPlugin()])
执行结果:
$ python myinvoke.py
*** test run reporting finishing
好的集成实践
- 使用虚拟环境
#virtualenv . New python executable in ./bin/python Installing setuptools, pip...done. root@AutoTest:[/data/code/python/pytest]#source bin/activate (pytest)root@AutoTest:[/data/code/python/pytest]#pip install pytest Downloading/unpacking pytest Downloading pytest-2.5.2.tar.gz (608kB): 608kB downloaded Running setup.py (path:/data/code/python/pytest/build/pytest/setup.py) egg_info for package pytest
Downloading/unpacking py>=1.4.20 (from pytest) Downloading py-1.4.22.tar.gz (189kB): 189kB downloaded Running setup.py (path:/data/code/python/pytest/build/py/setup.py) egg_info for package py
Installing collected packages: pytest, py Running setup.py install for pytest
Installing py.test-2.7 script to /data/code/python/pytest/bin
Installing py.test script to /data/code/python/pytest/bin
Running setup.py install for py
Successfully installed pytest py Cleaning up...
- 测试布局和导入规则
测试布局的方法有2种。一为放置在应用代码之外,适用于有很多功能测试等情况。
setup.py # your distutils/setuptools Python package metadata mypkg/ __init__.py appmodule.py tests/ test_app.py ...
二为嵌入测试目录到应用,当(单元)测试和应用之间的有直接关系,并想一起发布时有用:
setup.py # your distutils/setuptools Python package metadata mypkg/ __init__.py appmodule.py ... test/ test_app.py
部分使用pytest的项目
参见:Project examples
使用与实例
http://stackoverflow.com/search?q=pytest 上有很多例子。
断言
简单断言
# content of test_assert1.py
def f():
return 3
def test_function():
assert f() == 4
执行结果:
$ py.test test_assert1.py
=================================================================================== test session starts ====================================================================================
platform linux -- Python 3.5.1+, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /home/andrew/tmp, inifile:
plugins: xdist-1.14
collected 1 items
test_assert1.py F
========================================================================================= FAILURES =========================================================================================
______________________________________________________________________________________ test_function _______________________________________________________________________________________
def test_function():
> assert f() == 4
E assert 3 == 4
E + where 3 = f()
test_assert1.py:6: AssertionError
================================================================================= 1 failed in 0.00 seconds =================================================================================
$
-
异常断言
import pytest
def test_zero_division():
with pytest.raises(ZeroDivisionError):
1 / 0
获取异常信息:
def test_recursion_depth():
with pytest.raises(RuntimeError) as excinfo:
def f():
f()
f()
assert 'maximum recursion' in str(excinfo.value)
xfail中也可以指定异常类型:
import pytest
@pytest.mark.xfail(raises=NameError)
def test_f():
f()
pytest.raises适用于故意产生异常,@pytest.mark.xfail适用于处理bug。
上下文比较
# content of test_assert2.py def test_set_comparison(): set1 = set("1308") set2 = set("8035") assert set1 == set2
执行结果:
# py.test test_assert2.py ================================================================================================= test session starts ========================================================================= platform linux2 -- Python 2.7.5 -- py-1.4.23 -- pytest-2.6.1 collected 1 items
test_assert2.py F
============================================================================== FAILURES ============================================================================== ________________________________________________________________________ test_set_comparison _________________________________________________________________________
def test\_set\_comparison():
set1 = set("1308")
set2 = set("8035")
> assert set1 == set2 E assert set(['0', '1', '3', '8']) == set(['0', '3', '5', '8']) E Extra items in the left set: E '1' E Extra items in the right set: E '5'
test_assert2.py:5: AssertionError ====================================================================== 1 failed in 0.01 seconds ======================================================================
自定义断言比较
conftest.py
# content of conftest.py from test_foocompare import Foo def pytest_assertrepr_compare(op, left, right): if isinstance(left, Foo) and isinstance(right, Foo) and op == "==": return ['Comparing Foo instances:', 'vals: %s != %s' % (left.val, right.val)]
test_foocompare.py
# content of test_foocompare.py class Foo: def __init__(self, val): self.val = val
def test_compare(): f1 = Foo(1) f2 = Foo(2) assert f1 == f2
执行结果:
# py.test -q test_foocompare.py F ================================================================================================================ FAILURES ================================================================================================================= ______________________________________________________________________________________________________________ test_compare _______________________________________________________________________________________________________________
def test_compare():
f1 = Foo(1)
f2 = Foo(2)
> assert f1 == f2 E assert Comparing Foo instances: E vals: 1 != 2
test_foocompare.py:9: AssertionError 1 failed in 0.01 seconds
高级內省断言 ¶
暂略,后面补上
failure_demo.py 结合断言的使用,展示了多种错误报告:
https://github.com/alfredodeza/pytest/blob/master/doc/en/example/assertion/failure_demo.py
from py.test import raises
import py
def otherfunc(a,b):
assert a==b
def somefunc(x,y):
otherfunc(x,y)
def otherfunc_multi(a,b):
assert (a ==
b)
def test_generative(param1, param2):
assert param1 * 2 < param2
def pytest_generate_tests(metafunc):
if 'param1' in metafunc.fixturenames:
metafunc.addcall(funcargs=dict(param1=3, param2=6))
class TestFailing(object):
def test_simple(self):
def f():
return 42
def g():
return 43
assert f() == g()
def test_simple_multiline(self):
otherfunc_multi(
42,
6*9)
def test_not(self):
def f():
return 42
assert not f()
class TestSpecialisedExplanations(object):
def test_eq_text(self):
assert 'spam' == 'eggs'
def test_eq_similar_text(self):
assert 'foo 1 bar' == 'foo 2 bar'
def test_eq_multiline_text(self):
assert 'foo\nspam\nbar' == 'foo\neggs\nbar'
def test_eq_long_text(self):
a = '1'*100 + 'a' + '2'*100
b = '1'*100 + 'b' + '2'*100
assert a == b
def test_eq_long_text_multiline(self):
a = '1\n'*100 + 'a' + '2\n'*100
b = '1\n'*100 + 'b' + '2\n'*100
assert a == b
def test_eq_list(self):
assert [0, 1, 2] == [0, 1, 3]
def test_eq_list_long(self):
a = [0]*100 + [1] + [3]*100
b = [0]*100 + [2] + [3]*100
assert a == b
def test_eq_dict(self):
assert {'a': 0, 'b': 1, 'c': 0} == {'a': 0, 'b': 2, 'd': 0}
def test_eq_set(self):
assert set([0, 10, 11, 12]) == set([0, 20, 21])
def test_eq_longer_list(self):
assert [1,2] == [1,2,3]
def test_in_list(self):
assert 1 in [0, 2, 3, 4, 5]
def test_not_in_text_multiline(self):
text = 'some multiline\ntext\nwhich\nincludes foo\nand a\ntail'
assert 'foo' not in text
def test_not_in_text_single(self):
text = 'single foo line'
assert 'foo' not in text
def test_not_in_text_single_long(self):
text = 'head ' * 50 + 'foo ' + 'tail ' * 20
assert 'foo' not in text
def test_not_in_text_single_long_term(self):
text = 'head ' * 50 + 'f'*70 + 'tail ' * 20
assert 'f'*70 not in text
def test_attribute():
class Foo(object):
b = 1
i = Foo()
assert i.b == 2
def test_attribute_instance():
class Foo(object):
b = 1
assert Foo().b == 2
def test_attribute_failure():
class Foo(object):
def _get_b(self):
raise Exception('Failed to get attrib')
b = property(_get_b)
i = Foo()
assert i.b == 2
def test_attribute_multiple():
class Foo(object):
b = 1
class Bar(object):
b = 2
assert Foo().b == Bar().b
def globf(x):
return x+1
class TestRaises:
def test_raises(self):
s = 'qwe'
raises(TypeError, "int(s)")
def test_raises_doesnt(self):
raises(IOError, "int('3')")
def test_raise(self):
raise ValueError("demo error")
def test_tupleerror(self):
a,b = [1]
def test_reinterpret_fails_with_print_for_the_fun_of_it(self):
l = [1,2,3]
print ("l is %r" % l)
a,b = l.pop()
def test_some_error(self):
if namenotexi:
pass
def func1(self):
assert 41 == 42
# thanks to Matthew Scott for this test
def test_dynamic_compile_shows_nicely():
src = 'def foo():\n assert 1 == 0\n'
name = 'abc-123'
module = py.std.imp.new_module(name)
code = py.code.compile(src, name, 'exec')
py.builtin.exec_(code, module.__dict__)
py.std.sys.modules[name] = module
module.foo()
class TestMoreErrors:
def test_complex_error(self):
def f():
return 44
def g():
return 43
somefunc(f(), g())
def test_z1_unpack_error(self):
l = []
a,b = l
def test_z2_type_error(self):
l = 3
a,b = l
def test_startswith(self):
s = "123"
g = "456"
assert s.startswith(g)
def test_startswith_nested(self):
def f():
return "123"
def g():
return "456"
assert f().startswith(g())
def test_global_func(self):
assert isinstance(globf(42), float)
def test_instance(self):
self.x = 6*7
assert self.x != 42
def test_compare(self):
assert globf(10) < 5
def test_try_finally(self):
x = 1
try:
assert x == 0
finally:
x = 0
执行:
$ py.test test_failures.py
=================================================================================== test session starts ====================================================================================
platform linux -- Python 3.5.1, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /iscsi_data1/data/code/python/pytest/doc/en, inifile: pytest.ini
collected 1 items
test_failures.py .
================================================================================= 1 passed in 0.56 seconds =================================================================================
[root@public01 assertion]# py.test failure_demo.py
=================================================================================== test session starts ====================================================================================
platform linux -- Python 3.5.1, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /iscsi_data1/data/code/python/pytest/doc/en, inifile: pytest.ini
collected 39 items
failure_demo.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
========================================================================================= FAILURES =========================================================================================
____________________________________________________________________________________ test_generative[0] ____________________________________________________________________________________
param1 = 3, param2 = 6
def test_generative(param1, param2):
> assert param1 * 2 < param2
E assert (3 * 2) < 6
failure_demo.py:15: AssertionError
余下部分不做介绍,请以实际执行结果为准。
基本模式和实例
命令行传参数
下面函数中的cmdopt,需要从命令行读取:
# content of test_sample.py
def test_answer(cmdopt):
if cmdopt == "type1":
print ("first")
elif cmdopt == "type2":
print ("second")
assert 0 # to see what was printed
设置_conftest.py_
# content of conftest.py
import pytest
def pytest_addoption(parser):
parser.addoption("--cmdopt", action="store", default="type1",
help="my option: type1 or type2")
@pytest.fixture
def cmdopt(request):
return request.config.getoption("--cmdopt")
执行:
$ py.test -q --cmdopt=type2
F
========================================================================================= FAILURES =========================================================================================
_______________________________________________________________________________________ test_answer ________________________________________________________________________________________
cmdopt = 'type2'
def test_answer(cmdopt):
if cmdopt == "type1":
print ("first")
elif cmdopt == "type2":
print ("second")
> assert 0 # to see what was printed
E assert 0
test_sample.py:7: AssertionError
----------------------------------------------------------------------------------- Captured stdout call -----------------------------------------------------------------------------------
second
1 failed in 0.00 seconds
动态参数
# content of conftest.py
import sys
def pytest_cmdline_preparse(args):
if 'xdist' in sys.modules: # pytest-xdist plugin
import multiprocessing
num = max(multiprocessing.cpu_count() / 2, 1)
args[:] = ["-n", str(num)] + args
命令行忽略测试
下面用例,如果打了slow标签,则会需要添加选项--runslow才能执行。
配置
# content of conftest.py
import pytest
def pytest_addoption(parser):
parser.addoption("--runslow", action="store_true",
help="run slow tests")
用例:
# content of test_module.py
import pytest
slow = pytest.mark.skipif(
not pytest.config.getoption("--runslow"),
reason="need --runslow option to run"
)
def test_func_fast():
pass
@slow
def test_func_slow():
pass
默认执行:
$ py.test -rs
=================================================================================== test session starts ====================================================================================
platform linux -- Python 3.5.1+, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /home/andrew/tmp, inifile:
plugins: xdist-1.14
collected 2 items
test_module.py .s
================================================================================= short test summary info ==================================================================================
SKIP [1] test_module.py:15: need --runslow option to run
=========================================================================== 1 passed, 1 skipped in 0.01 seconds ============================================================================
$
指定标签执行
$ py.test --runslow
=================================================================================== test session starts ====================================================================================
platform linux -- Python 3.5.1+, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /home/andrew/tmp, inifile:
plugins: xdist-1.14
collected 2 items
test_module.py ..
================================================================================= 2 passed in 0.01 seconds =================================================================================
$
标签
Fixture
python中经典的xunit风格的setup是2005年首先出现在pytest,后面被nose和unittest采用。建议优先使用fixture,因为它有依赖注入,更强大。
- 模块级别
def setup_module(module):
""" setup any state specific to the execution of the given module."""
def teardown_module(module):
""" teardown any state that was previously setup with a setup_module
method.
"""
- 类级别
@classmethod
def setup_class(cls):
""" setup any state specific to the execution of the given class (which
usually contains tests).
"""
@classmethod
def teardown_class(cls):
""" teardown any state that was previously setup with a call to
setup_class.
"""
- 方法和函数级别
@classmethod
def setup_class(cls):
""" setup any state specific to the execution of the given class (which
usually contains tests).
"""
@classmethod
def teardown_class(cls):
""" teardown any state that was previously setup with a call to
setup_class.
"""
def setup_function(function):
""" setup any state tied to the execution of the given function.
Invoked for every test function in the module.
"""
def teardown_function(function):
""" teardown any state that was previously setup with a setup_function
call.
"""
临时目录和文件
tmpdir fixture
tmpdir是py.path.local对象,提供了os.path等的方法。
test_tmpdir.py
# content of test_tmpdir.py
import os
def test_create_file(tmpdir):
p = tmpdir.mkdir("sub").join("hello.txt")
p.write("content")
assert p.read() == "content"
assert len(tmpdir.listdir()) == 1
assert 0
执行结果:
$ py.test test_tmpdir.py
======================================================================= test session starts =======================================================================
platform linux -- Python 3.5.1+, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /home/andrew/tmp/py_test, inifile:
collected 1 items
test_tmpdir.py F
============================================================================ FAILURES =============================================================================
________________________________________________________________________ test_create_file _________________________________________________________________________
tmpdir = local('/tmp/pytest-of-andrew/pytest-0/test_create_file0')
def test_create_file(tmpdir):
p = tmpdir.mkdir("sub").join("hello.txt")
p.write("content")
assert p.read() == "content"
assert len(tmpdir.listdir()) == 1
> assert 0
E assert 0
test_tmpdir.py:8: AssertionError
==================================================================== 1 failed in 0.06 seconds =====================================================================
$
生成的目录一般在系统临时目录下面,格式为pytest-NUM,比如/tmp/pytest-of-andrew/pytest-0/test_create_file0,每次测试执行时NUM会增加,3个以前的目录会清除。如下方式会指定为其他目录:py.test --basetemp=mydir。
tmpdir_factory fixture
2.8新增
tmpdir_factory基于session,可创建任意临时目录。
例如,假设test suite需要大文件:
# contents of conftest.py
import pytest
@pytest.fixture(scope='session')
def image_file(tmpdir_factory):
img = compute_expensive_image()
fn = tmpdir_factory.mktemp('data').join('img.png')
img.save(str(fn))
return fn
# contents of test_image.py
def test_histogram(image_file):
img = load_image(image_file)
# compute and test histogram
相关方法:
TempdirFactory.``mktemp
(basename, numbered=True)[source]
TempdirFactory.``getbasetemp
()[source]
捕捉输出
默认stdout/stderr/stdin捕捉行为
默认捕捉stdout/stderr,如果test或者setup方法失败,traceback会打印相关内容。
stdin为null,读取会报错,因为自动化测试鲜有交互式输入。
默认捕捉为写入到底层文件,可以捕捉print语言输出到test中的subprocess输出。
设置捕捉
默认捕捉方式为file descriptor (FD)级捕捉。捕捉所有到操作系统的1,2输出。
syslevel级捕捉只捕捉python的sys.stdout和sys.stderr。
py.test -s # disable all capturing
py.test --capture=sys # replace sys.stdout/stderr with in-mem files
py.test --capture=fd # also point filedescriptors 1 and 2 to temp file
使用print进行调试
默认捕捉stdout/stderr:
# content of test_module.py
def setup_function(function):
print ("setting up %s" % function)
def test_func1():
assert True
def test_func2():
assert False
执行结果:
$ py.test test_module.py
======================================================================= test session starts =======================================================================
platform linux -- Python 3.5.1+, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /home/andrew/tmp/py_test, inifile:
collected 2 items
test_module.py .F
============================================================================ FAILURES =============================================================================
___________________________________________________________________________ test_func2 ____________________________________________________________________________
def test_func2():
> assert False
E assert False
test_module.py:10: AssertionError
---------------------------------------------------------------------- Captured stdout setup ----------------------------------------------------------------------
setting up
=============================================================== 1 failed, 1 passed in 0.11 seconds ================================================================
$