How can I access the overall test result of a pytest test run during runtime?

by

I could not find a suitable pytest hook to access the overall test result yet.

You don’t need one; just collect the test results yourself. This is the blueprint I usually use when in need of accessing the test results in batch:

# conftest.py
import pytest


def pytest_sessionstart(session):
    session.results = dict()


@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    result = outcome.get_result()

    if result.when == 'call':
        item.session.results[item] = result

Now all test results are stored under session.results dict; example usage:

# conftest.py (continued)

def pytest_sessionfinish(session, exitstatus):
    print()
    print('run status code:', exitstatus)
    passed_amount = sum(1 for result in session.results.values() if result.passed)
    failed_amount = sum(1 for result in session.results.values() if result.failed)
    print(f'there are {passed_amount} passed and {failed_amount} failed tests')

Running the tests will yield:

$ pytest -sv
================================== test session starts ====================================
platform darwin -- Python 3.6.4, pytest-3.7.1, py-1.5.3, pluggy-0.7.1 -- /Users/hoefling/.virtualenvs/stackoverflow/bin/python3.6
cachedir: .pytest_cache
rootdir: /Users/hoefling/projects/private/stackoverflow/so-51711988, inifile:
collected 3 items

test_spam.py::test_spam PASSED
test_spam.py::test_eggs PASSED
test_spam.py::test_fail FAILED
run status code: 1
there are 2 passed and 1 failed tests


======================================== FAILURES =========================================
_______________________________________ test_fail _________________________________________

    def test_fail():
>       assert False
E       assert False

test_spam.py:10: AssertionError
=========================== 1 failed, 2 passed in 0.05 seconds ============================

In case the overall pytest exit code (exitstatus) is sufficient info (info about # passed, # failed, etc. not required) use the following:

# conftest.py

def pytest_sessionfinish(session, exitstatus):
    print()
    print('run status code:', exitstatus)

Accessing the error

You can access the error details from the call.excinfo object:

@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    result = outcome.get_result()
    if result.outcome == "failed":
        exception = call.excinfo.value
        exception_class = call.excinfo.type
        exception_class_name = call.excinfo.typename
        exception_type_and_message_formatted = call.excinfo.exconly()
        exception_traceback = call.excinfo.traceback

More Related Contents:

  1. PATH issue with pytest ‘ImportError: No module named YadaYadaYada’
  2. How to pass arguments in pytest by command line
  3. How to properly assert that an exception gets raised in pytest?
  4. Pass a parameter to a fixture function
  5. In pytest, what is the use of conftest.py files?
  6. Using pytest with a src layer
  7. How to test a function with input call?
  8. Test case execution order in pytest
  9. How do I configure PyCharm to run py.test tests?
  10. How to parametrize a Pytest fixture
  11. How to skip the rest of tests in the class if one has failed?
  12. Mocking a module import in pytest
  13. Pytest fixture for a class through self not as method argument
  14. How can I see normal print output created during pytest run?
  15. Logging within pytest tests
  16. pytest cannot import module while python can
  17. How do I correctly setup and teardown for my pytest class with tests?
  18. How to monkeypatch python’s datetime.datetime.now with py.test?
  19. pytest cannot find module [duplicate]
  20. Is there a way to specify which pytest tests to run from a file?
  21. How to suppress py.test internal deprecation warnings
  22. pytest using fixtures as arguments in parametrize
  23. py.test skips test class if constructor is defined
  24. VSCode pytest test discovery fails
  25. how to share a variable across modules for all tests in py.test
  26. Selenium – visibility_of_element_located: __init__() takes exactly 2 arguments (3 given)
  27. How can I limit the maximum running time for a unit test?
  28. In which order are pytest fixtures executed?
  29. PyCharm noinspection for whole file?
  30. How to mock a readonly property with mock?

Leave a Comment