[pytest] 配置

Pytest

Init

Run

1. 更改配置pytest.ini与项目同级

# content of pytest.ini
# Example 1: have pytest look for "check" instead of "test"
[pytest]
;更改目录递归
norecursedirs = .svn _build tmp*

;更改命名约定
python_files = check_*.py
python_classes = Check
python_functions = *_check
;可以通过在模式之间添加空格来检查多个 glob 模式
;python_files = test_*.py example_*.py

;将命令行参数解释
;addopts = --tb=short
;addopts = --pyargs
;export PYTEST_ADDOPTS="-v"
addopts = -vv --html-report=report.html

1. 引进@pytest.mark.parametrize中ids导致编码乱码

def pytest_collection_modifyitems(items):
    for item in items:
        item.name = item.name.encode('utf-8').decode('unicode-escape')
        item._nodeid = item._nodeid.encode('utf-8').decode('unicode-escape')

2. 定义自己对失败断言的解释 pytest_assertrepr_compare(config, op, left, right)

   • config (Config) – The pytest config object.
   • op (str) – The operator, e.g. “==”, “!=”, “not in”.
   • left (object) – The left operand.
   • right (object) – The right operand.

# content of conftest.py
from test_foocompare import Foo


def pytest_assertrepr_compare(op, left, right):
    if isinstance(left, Foo) and isinstance(right, Foo) and op == "==":
        return [
            "Comparing Foo instances:",
            f"   vals: {left.val} != {right.val}",
        ]

# content of test_foocompare.py
class Foo:
    def __init__(self, val):
        self.val = val

    def __eq__(self, other):
        return self.val == other.val


def test_compare():
    f1 = Foo(1)
    f2 = Foo(2)
    assert f1 == f2

output

$ pytest -q test_foocompare.py
F                                                                    [100%]
================================= FAILURES =================================
_______________________________ test_compare _______________________________

    def test_compare():
        f1 = Foo(1)
        f2 = Foo(2)
>       assert f1 == f2
E       assert Comparing Foo instances:
E            vals: 1 != 2

test_foocompare.py:12: AssertionError
========================= short test summary info ==========================
FAILED test_foocompare.py::test_compare - assert Comparing Foo instances:
1 failed in 0.12s

3. 根据命令行选项将不同的值传递给测试函数

# content of conftest.py
import pytest


def pytest_addoption(parser):
    parser.addoption(
        "--cmdopt", action="store", default="type1", help="my option: type1 or type2"
    )


@pytest.fixture
def cmdopt(request):
    return request.config.getoption("--cmdopt")

# content of test_sample.py
def test_answer(cmdopt):
    if cmdopt == "type1":
        print("first")
    elif cmdopt == "type2":
        print("second")
    assert 0  # to see what was printed

output

# ************
# 没有提供参数
# ************
$ pytest -q test_sample.py 
F                                                                    [100%]
================================= FAILURES =================================
_______________________________ test_answer ________________________________

cmdopt = 'type1'

    def test_answer(cmdopt):
        if cmdopt == "type1":
            print("first")
        elif cmdopt == "type2":
            print("second")
>       assert 0  # to see what was printed
E       assert 0

test_sample.py:6: AssertionError
--------------------------- Captured stdout call ---------------------------
first
========================= short test summary info ==========================
FAILED test_sample.py::test_answer - assert 0
1 failed in 0.12s

# ************
# 提供参数
# ************
$ pytest -q --cmdopt=type2
F                                                                    [100%]
================================= FAILURES =================================
_______________________________ test_answer ________________________________

cmdopt = 'type2'

    def test_answer(cmdopt):
        if cmdopt == "type1":
            print("first")
        elif cmdopt == "type2":
            print("second")
>       assert 0  # to see what was printed
E       assert 0

test_sample.py:6: AssertionError
--------------------------- Captured stdout call ---------------------------
second
========================= short test summary info ==========================
FAILED test_sample.py::test_answer - assert 0
1 failed in 0.12s

• 如果需要更详细信息

# content of conftest.py
import pytest


def type_checker(value):
    msg = "cmdopt must specify a numeric type as typeNNN"
    if not value.startswith("type"):
        raise pytest.UsageError(msg)
    try:
        int(value[4:])
    except ValueError:
        raise pytest.UsageError(msg)

    return value


def pytest_addoption(parser):
    parser.addoption(
        "--cmdopt",
        action="store",
        default="type1",
        help="my option: type1 or type2",
        type=type_checker,
    )

output

$ pytest -q --cmdopt=type3
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: argument --cmdopt: invalid choice: 'type3' (choose from 'type1', 'type2')

Report

1. 向测试报告标题添加信息

1.1

# content of conftest.py


def pytest_report_header(config):
    return "project deps: mylib-1.1"

output

$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
project deps: mylib-1.1
rootdir: /home/sweet/project
collected 0 items

========================== no tests ran in 0.12s ===========================

1.2 返回字符串列表，这些字符串将被视为多行信息

# content of conftest.py


def pytest_report_header(config):
    if config.get_verbosity() > 0:
        return ["info1: did you know that ...", "did you?"]

output 仅在使用“-v”运行时才会添加信息

$ pytest -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
info1: did you know that ...
did you?
rootdir: /home/sweet/project
collecting ... collected 0 items

========================== no tests ran in 0.12s ===========================

2. 分析测试持续时间 pytest --durations=3

# content of test_some_are_slow.py
import time


def test_funcfast():
    time.sleep(0.1)


def test_funcslow1():
    time.sleep(0.2)


def test_funcslow2():
    time.sleep(0.3)

output

$ pytest --durations=3
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 3 items

test_some_are_slow.py ...                                            [100%]

=========================== slowest 3 durations ============================
0.30s call     test_some_are_slow.py::test_funcslow2
0.20s call     test_some_are_slow.py::test_funcslow1
0.10s call     test_some_are_slow.py::test_funcfast
============================ 3 passed in 0.12s =============================

3. 增量测试 - 测试步骤

如果前置步骤其中一个步骤失败，则后续步骤将预期失败。

# content of conftest.py

from typing import Dict, Tuple

import pytest

# store history of failures per test class name and per index in parametrize (if parametrize used)
_test_failed_incremental: Dict[str, Dict[Tuple[int, ...], str]] = {}


def pytest_runtest_makereport(item, call):
    if "incremental" in item.keywords:
        # incremental marker is used
        if call.excinfo is not None:
            # the test has failed
            # retrieve the class name of the test
            cls_name = str(item.cls)
            # retrieve the index of the test (if parametrize is used in combination with incremental)
            parametrize_index = (
                tuple(item.callspec.indices.values())
                if hasattr(item, "callspec")
                else ()
            )
            # retrieve the name of the test function
            test_name = item.originalname or item.name
            # store in _test_failed_incremental the original name of the failed test
            _test_failed_incremental.setdefault(cls_name, {}).setdefault(
                parametrize_index, test_name
            )


def pytest_runtest_setup(item):
    if "incremental" in item.keywords:
        # retrieve the class name of the test
        cls_name = str(item.cls)
        # check if a previous test has failed for this class
        if cls_name in _test_failed_incremental:
            # retrieve the index of the test (if parametrize is used in combination with incremental)
            parametrize_index = (
                tuple(item.callspec.indices.values())
                if hasattr(item, "callspec")
                else ()
            )
            # retrieve the name of the first test function to fail for this class name and index
            test_name = _test_failed_incremental[cls_name].get(parametrize_index, None)
            # if name found, test has failed for the combination of class name & test name
            if test_name is not None:
                pytest.xfail(f"previous test failed ({test_name})")

# content of test_step.py

import pytest


@pytest.mark.incremental
class TestUserHandling:
    def test_login(self):
        pass

    def test_modification(self):
        assert 0

    def test_deletion(self):
        pass


def test_normal():
    pass

output

$ pytest -rx
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 4 items

test_step.py .Fx.                                                    [100%]

================================= FAILURES =================================
____________________ TestUserHandling.test_modification ____________________

self = <test_step.TestUserHandling object at 0xdeadbeef0001>

    def test_modification(self):
>       assert 0
E       assert 0

test_step.py:11: AssertionError
========================= short test summary info ==========================
XFAIL test_step.py::TestUserHandling::test_deletion - reason: previous test failed (test_modification)
================== 1 failed, 2 passed, 1 xfailed in 0.12s ==================

–junitxml={report}.xml

https://docs.pytest.org/en/stable/reference/reference.html
pytest v6.0+ 默认 xunit2 不支持 testcase添加属性
建议设置

• pytest.ini中配置junit_family=xunit1
• pytest -o junit_family=xunit1

<?xml version="1.0" encoding="utf-8"?>
<testsuites>
    <testsuite name="pytest" errors="0" failures="0" skipped="0" tests="2" time="0.113"
               timestamp="2025-03-10T14:53:08.040765+08:00" hostname="Ding-Perlis-MP1Y70F1">
        <testcase classname="set_classname" name="set_name" file="test_case.py" line="49" time="0.006"/>
        <testcase classname="set_classname" name="set_name" file="test_case.py" line="49" time="0.001"/>
    </testsuite>
</testsuites>

1. testsuite

1.1 在测试套件级别添加属性节点 record_testsuite_property

支持xunit2

import pytest


@pytest.fixture(scope="session", autouse=True)
def log_global_env_facts(record_testsuite_property):
    record_testsuite_property("ARCH", "PPC")
    record_testsuite_property("STORAGE_TYPE", "CEPH")


class TestMe:
    def test_foo(self):
        assert True

output

<testsuite errors="0" failures="0" name="pytest" skipped="0" tests="1" time="0.006">
    <properties>
        <property name="ARCH" value="PPC"/>
        <property name="STORAGE_TYPE" value="CEPH"/>
    </properties>
    <testcase classname="test_me.TestMe" file="test_me.py" line="16" name="test_foo" time="0.000243663787842"/>
</testsuite>

2. testcase

2.1 记录测试的其他信息 record_property

请注意，使用此功能将中断对最新JUnitXML架构的架构验证。当与某些CI服务器一起使用时，这可能是一个问题

• 方法一 test_case.py

def test_function(record_property):
    record_property("example_key", 1)
    assert True

• 方法二 contest.py

# content of conftest.py

def pytest_collection_modifyitems(session, config, items):
    for item in items:
        for marker in item.iter_markers(name="test_id"):
            test_id = marker.args[0]
            item.user_properties.append(("test_id", test_id))


# content of test_function.py
import pytest


@pytest.mark.test_id(1501)
def test_function():
    assert True

output

<templt>
    <testcase classname="test_function" file="test_function.py" line="0" name="test_function" time="0.0009">
        <properties>
            <property name="example_key" value="1"/>
        </properties>
    </testcase>
    <testcase classname="test_function" file="test_function.py" line="0" name="test_function" time="0.0009">
        <properties>
            <property name="test_id" value="1501"/>
        </properties>
    </testcase>
</templt>

2.2 向testcase元素添加额外的xml属性 record_xml_attribute

record_xml_attribute 是一个实验性的特性，它的接口在未来的版本中可能会被更强大和通用的东西所取代。然而，功能本身将保持不变
请注意，使用此功能将中断对最新JUnitXML架构的架构验证。当与某些CI服务器一起使用时，这可能是一个问题

• 方法一 test_case.py

import pytest


@pytest.mark.parametrize("case", ["case1", "case2"])
def test_case(case, record_xml_attribute):
    record_xml_attribute('classname', 'set_classname')  # 重写 value
    record_xml_attribute('name', 'set_name')  # 重写 value
    record_xml_attribute('index', '123')  # 新增 key, value
    print("hello world")
    assert True

• 方法二 contest.py

# edit to contest.py
import pytest


@pytest.fixture(autouse=True)
def record_index(record_xml_attribute):
    record_xml_attribute('index', '123')  # 新增 key, value

• output

<?xml version="1.0" encoding="utf-8"?>
<testsuites>
    <testsuite name="pytest" errors="0" failures="0" skipped="0" tests="2" time="0.113"
               timestamp="2025-03-10T14:53:08.040765+08:00" hostname="Ding-Perlis-MP1Y70F1">
        <testcase classname="set_classname" name="set_name" file="test_case.py" line="49" index="123" time="0.006">
            <system-out>
                hello world
            </system-out>
        </testcase>
        <testcase classname="set_classname" name="set_name" file="test_case.py" line="49" index="123" time="0.001"/>
    </testsuite>
</testsuites>

Hooks

other plugin 好玩好用的

• pytest_html_merger https://github.com/akavbathen/pytest_html_merger
  pip install pytest_html_merger 合并pytest_html报告

  export PATH="$HOME/.lcoal/bin:$PATH"
  pytest_html_merger -i /path/to/your/html/reports -o /path/to/output/report/merged.html
  

• pytest-tally https://github.com/jeffwright13/pytest-tally
  pip install pytest-tally可在控制台、应用程序或浏览器中显示测试运行进度

  cd project # 与main.py同级
  python main.py
  pytest xxx
  
  # tally
  tally-rich
  tally-flask
  tally-tk
  

  
• pytest-sugarhttps://pypi.org/project/pytest-sugar/
  pip install pytest-sugar改变 pytest 的默认外观（例如进度条、立即显示失败的测试）