Description
When an internal error occurs inside the pytest_runtest_makereport hook,
pytest-html 3.1.1 does not receive a valid report outcome and silently
defaults to displaying the test as Passed in the HTML report.
Environment
- pytest-html: 3.1.1
- pytest: 7.1.2 or 9.0.3
- Python: 3.13.7
- OS: Windows
Expected Behavior
Test should be marked as ERROR in the HTML report.
Actual Behavior
Test is marked as Passed in the HTML report, giving a false green result.
Steps to Reproduce
- Trigger an exception inside
pytest_runtest_makereport hook for eg raise RuntimeError("BOOM")
- Run pytest with
--html=report.html
- Open the HTML report
- Observe the affected test shows as Passed
Additional Notes
pytest-html 4.2.0 handles this differently — the report is empty instead
of showing a false Passed — but neither behavior is correct.
the correct expected outcome is debatable:
ERROR — represents a framework/fixture level problem, not a test assertion failure. This is probably the most accurate.
FAILED — could argue this since something clearly went wrong.
At minimum — not Passed, since that's factually wrong.
-In case of keyboard interrupt too same issue
Description
When an internal error occurs inside the
pytest_runtest_makereporthook,pytest-html 3.1.1 does not receive a valid report outcome and silently
defaults to displaying the test as Passed in the HTML report.
Environment
Expected Behavior
Test should be marked as ERROR in the HTML report.
Actual Behavior
Test is marked as Passed in the HTML report, giving a false green result.
Steps to Reproduce
pytest_runtest_makereporthook for eg raise RuntimeError("BOOM")--html=report.htmlAdditional Notes
pytest-html 4.2.0 handles this differently — the report is empty instead
of showing a false Passed — but neither behavior is correct.
the correct expected outcome is debatable:
ERROR — represents a framework/fixture level problem, not a test assertion failure. This is probably the most accurate.
FAILED — could argue this since something clearly went wrong.
At minimum — not Passed, since that's factually wrong.
-In case of keyboard interrupt too same issue