Hide details of xpass, but keep the short list of them in "short test summary info"

[cbeauchesne]

Hi!

I'm moving from 7.x to 8.x and I've noticed that the -r X option now prints the logs of xpasses in stdout. #11574

Is there a way to get the old behavior : list of xpasses shown in short test summary info, but not more details ?

My use case is that my test logs are very verbose, and adding them for xpasses by default in stdout is a no go, as it makes other details on failure harder to reach (my users are very sensitives on the signal/noise ratio in stdout).

Though, I'd like to keep the list of xpass tests, otherwise, they are invisibilised, and it reduces the incentive to activate them (this short list was the sweet spot in term of visibility/noise).

[Labeeb2339]

Use the targeted capture option together with -rX:

pytest -q -rX --show-capture=no

-rX still adds each XPASS to the short test summary info, while --show-capture=no prevents the captured stdout/stderr/log sections from being printed in the detailed XPASSES block. You can put the same values in addopts in pytest.ini or pyproject.toml if this should be the default.

--tb=no is broader: it suppresses traceback output globally, so --show-capture=no is the better fit when the goal is specifically to reduce XPASS noise.

[cbeauchesne]

Checking right now

[cbeauchesne]

Unfortunately, --show-capture=no is not a good fit : indeed it suppress logs capture for xpasses. But also for failures, which is a key data in stdout.

[SirHegel]

There is no option for this yet. -rX is doing two things at once since 8.0: it adds the XPASS lines to short test summary info, and it enables the XPASSES section that dumps captured output for each xpassed test. In _pytest/terminal.py, summary_xpasses() is summary_passes_combined("xpassed", "XPASSES", "X"), and the only two things that method checks are --tb=no and whether X is in the report chars. --show-capture=no is not a fit for the reason you found: _outrep_summary() applies it to the FAILURES section as well.

For xfail the same complaint was raised in #12231 and pytest 8.3 detached the tracebacks from -rx behind a separate --xfail-tb flag. Nothing equivalent exists for xpass, so a request for an analogous switch would be consistent with that precedent.

Until then, a conftest.py can turn the section off while keeping the short summary. summary_xpasses is invoked by the terminal reporter's own pytest_terminal_summary wrapper, and short_test_summary() runs separately afterwards, so replacing the method on the reporter instance only removes the detailed block:

# conftest.py
def pytest_sessionstart(session):
    reporter = session.config.pluginmanager.getplugin("terminalreporter")
    if reporter is not None:
        # Keep the XPASS lines from -rX in "short test summary info",
        # drop the XPASSES section with captured output (pytest >= 8.0).
        reporter.summary_xpasses = lambda: None

Checked with pytest -q -rX on 8.3.5 and 9.1.1: the FAILURES section still includes captured stdout/log for failures, XPASS test_x.py::test_xpass - flaky still appears in the short summary, and the XPASSES block is gone. pytest_sessionstart is used rather than a plain pytest_configure because the reporter is registered by pytest's own pytest_configure implementation, and conftest hook implementations run before it (checked: getplugin("terminalreporter") returns None there). A @pytest.hookimpl(trylast=True) pytest_configure also works if you prefer to keep it in configure.

It is a private method, so treat it as a workaround pinned to the versions you run, not as API.