Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/1164.added.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Added the ``pytest_asyncio_loop_factories`` hook to parametrize asyncio tests with custom event loop factories.
46 changes: 46 additions & 0 deletions docs/how-to-guides/custom_loop_factory.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
==================================================
How to use custom event loop factories for tests
==================================================

pytest-asyncio can run asynchronous tests with custom event loop factories by defining a ``pytest_asyncio_loop_factories`` hook in ``conftest.py``. The hook returns the factories to use for the current test item:

.. code-block:: python

import asyncio

import pytest


class CustomEventLoop(asyncio.SelectorEventLoop):
pass


def pytest_asyncio_loop_factories(config, item):
return [CustomEventLoop]

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The content of this document is great. I just think it should be split over several documents to be consistent with the existing structutre.

Pytest-asyncio roughly follows the Diataxis documentation framework. I suggest to stop the How-to for custom loop factories right here at line 20, move 25–38 into a dedicated how-to guide, and pack the rest of the information into the reference section of the docs.

When multiple factories are returned, each asynchronous test is run once per factory. Synchronous tests are not parametrized. The configured loop scope still determines how long each event loop instance is kept alive.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some test suites that use other plugins for async tests besides pytest-asyncio. Since pytest-trio/pytest-anyio and pytest-asyncio can coexist peacefully, the phrasing "each asynchronous test is run once per factory" made me wonder if this also affects async tests from other plugins.

The answer should be obvious, but to clear out any doubt, we could say "each pytest-asyncio test is run once per factory". Or we add another sentence that this doesn't affect tests not managed by pytest-asyncio.


Factories should be callables without required parameters and should return an ``asyncio.AbstractEventLoop`` instance. The hook must return a non-empty sequence for every asyncio test.

When multiple ``pytest_asyncio_loop_factories`` implementations are present, pytest-asyncio uses the first non -``None`` result in pytest's normal hook dispatch order.

To select different factories for specific tests, you can inspect ``item``:

.. code-block:: python

import asyncio

import uvloop


def pytest_asyncio_loop_factories(config, item):
if item.get_closest_marker("uvloop"):
return [uvloop.new_event_loop]
else:
return [asyncio.new_event_loop]

Factory selection can vary per test item, regardless of loop scope. In other words, with ``module``/``package``/``session`` loop scopes you can still choose different factories for different tests by inspecting ``item``.

.. note::

When the hook is defined, async tests are parametrized, so factory names are appended to test IDs. For example, a test ``test_example`` with factory ``CustomEventLoop`` will appear as ``test_example[CustomEventLoop]`` in the test output.
1 change: 1 addition & 0 deletions docs/how-to-guides/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ How-To Guides
change_fixture_loop
change_default_fixture_loop
change_default_test_loop
custom_loop_factory
run_class_tests_in_same_loop
run_module_tests_in_same_loop
run_package_tests_in_same_loop
Expand Down
25 changes: 23 additions & 2 deletions docs/how-to-guides/uvloop.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,29 @@
How to test with uvloop
=======================

Redefining the *event_loop_policy* fixture will parametrize all async tests. The following example causes all async tests to run multiple times, once for each event loop in the fixture parameters:
Replace the default event loop policy in your *conftest.py:*
Define a ``pytest_asyncio_loop_factories`` hook in your *conftest.py* that returns ``uvloop.new_event_loop`` as a loop factory:

.. code-block:: python

import uvloop


def pytest_asyncio_loop_factories(config, item):
return [uvloop.new_event_loop]

.. seealso::

:doc:`custom_loop_factory`
More details on the ``pytest_asyncio_loop_factories`` hook, including per-test factory selection and multiple factory parametrization.

Using the event_loop_policy fixture
------------------------------------

.. note::

``asyncio.AbstractEventLoopPolicy`` is deprecated as of Python 3.14 (removal planned for 3.16), and ``uvloop.EventLoopPolicy`` will be removed alongside it. Prefer the hook approach above.

For older versions of Python and uvloop, you can override the *event_loop_policy* fixture in your *conftest.py:*

.. code-block:: python

Expand Down
80 changes: 79 additions & 1 deletion pytest_asyncio/plugin.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
Any,
Literal,
ParamSpec,
TypeAlias,
TypeVar,
overload,
)
Expand Down Expand Up @@ -63,6 +64,7 @@
_R = TypeVar("_R", bound=Awaitable[Any] | AsyncIterator[Any])
_P = ParamSpec("_P")
FixtureFunction = Callable[_P, _R]
LoopFactory: TypeAlias = Callable[[], AbstractEventLoop]


class PytestAsyncioError(Exception):
Expand All @@ -74,6 +76,19 @@ class Mode(str, enum.Enum):
STRICT = "strict"


hookspec = pluggy.HookspecMarker("pytest")


class PytestAsyncioSpecs:
@hookspec
def pytest_asyncio_loop_factories(
self,
config: Config,
item: Item,
) -> Iterable[LoopFactory] | None:
raise NotImplementedError # pragma: no cover


ASYNCIO_MODE_HELP = """\
'auto' - for automatically handling all async functions by the plugin
'strict' - for autoprocessing disabling (useful if different async frameworks \
Expand All @@ -83,6 +98,7 @@ class Mode(str, enum.Enum):


def pytest_addoption(parser: Parser, pluginmanager: PytestPluginManager) -> None:
pluginmanager.add_hookspecs(PytestAsyncioSpecs)
group = parser.getgroup("asyncio")
group.addoption(
"--asyncio-mode",
Expand Down Expand Up @@ -219,6 +235,38 @@ def _get_asyncio_debug(config: Config) -> bool:
return val == "true"


def _collect_hook_loop_factories(
config: Config,
item: Item,
) -> tuple[LoopFactory, ...] | None:
hook_caller = config.hook.pytest_asyncio_loop_factories
if not hook_caller.get_hookimpls():
return None

results: list[Iterable[LoopFactory] | None] = hook_caller(config=config, item=item)
msg = "pytest_asyncio_loop_factories must return a non-empty sequence of callables."
if not results:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since _collect_hook_loop_factories is run for virtually every test function, the following checks are also executed every time. My understanding is that this happens during collection time, because the exact set of test items isn't known before pytest_generate_tests finishes.

I don't think we need to take immediate action here. I merely want to point out that we should keep an eye on whether this has any significant impact on test collection performance. This can make life harder for users with large test suites.

raise pytest.UsageError(msg)
result = results[0]
if result is None or not isinstance(result, Sequence):
raise pytest.UsageError(msg)
# Copy into an immutable snapshot so later mutations of the hook's
# original container do not affect stash state or parametrization.
factories = tuple(result)
if not factories or any(not callable(factory) for factory in factories):
raise pytest.UsageError(msg)
return factories


def _get_item_loop_scope(item: Item, config: Config) -> _ScopeName:
marker = item.get_closest_marker("asyncio")
default_loop_scope = _get_default_test_loop_scope(config)
if marker is None:
return default_loop_scope
else:
return _get_marked_loop_scope(marker, default_loop_scope)


_DEFAULT_FIXTURE_LOOP_SCOPE_UNSET = """\
The configuration option "asyncio_default_fixture_loop_scope" is unset.
The event loop scope for asynchronous fixtures will default to the "fixture" caching \
Expand Down Expand Up @@ -611,6 +659,27 @@ def pytest_pycollect_makeitem_convert_async_functions_to_subclass(
hook_result.force_result(updated_node_collection)


@pytest.hookimpl(tryfirst=True)
def pytest_generate_tests(metafunc: pytest.Metafunc) -> None:
if _get_asyncio_mode(
metafunc.config
) == Mode.STRICT and not metafunc.definition.get_closest_marker("asyncio"):
return
if PytestAsyncioFunction.item_subclass_for(metafunc.definition) is None:
return
hook_factories = _collect_hook_loop_factories(metafunc.config, metafunc.definition)
if hook_factories is None:
return
metafunc.fixturenames.append(_asyncio_loop_factory.__name__)
loop_scope = _get_item_loop_scope(metafunc.definition, metafunc.config)
metafunc.parametrize(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's say a configured configured pytest_asyncio_loop_factories so that all tests to run with uvloop and SelectorEventLoop. There's one specific test that should only be run with uvloop. This means the user would have to switch to the conftest.py file and find some way to identify the test (e.g. custom marker or item name) to tell pytest-asyncio to use a different set of factories for this test.

Ergonomically, it would be preferable for the user to perform this change directly at the level of the test without having to change the hook implementation. This would also improve visibility that this test is an exception.

Based on the implementation of _get_item_loop_scope my understanding is that we have access to the asyncio marker here. Could we use the information from the marker to modify the test parametrization?

I'm thinking of something like this:

def pytest_asyncio_loop_factories(config, item):
    return {
        "default": asyncio.new_event_loop,
        "uvloop": uvloop.new_event_loop,
    }

async def parametrized_over_all_factories():
    ...

@pytest.mark.asyncio(loop_factories=["uvloop"])
async def  only_runs_with_uvloop():
    ...

@pytest.mark.asyncio(loop_factories=["default"])
async def  only_runs_with_default_asyncio_loop():
    ...

Do you think this is feasible and/or reasonable? I'd love to hear your opinion on this @tjkuson .

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seifertm I can see that being easier. The workflow I had in mind was that people would configure their own marks to achieve this (e.g., designing a pytest.mark.uvloop mark), but that can get complex, especially if there are lots of loop factory permutations.

The main downsides I can think of to the alternative approach you're suggesting:

  • We now have more than one way to do the same thing, which could be confusing to communicate
  • The mapping structure might be misleading (e.g., we don't usually provide mappings where the default behaviour is to collect all of its values), whereas it should be intuitive that all things returned by an ordinary collection are used to parametrize tests
  • pytest-asyncio now carries additional complexity that needs to be maintained (namely due to the additional marker check upon test collection) versus just letting it be the user's responsibility

I think the benefit to the developer experience outweighs these negatives, however, so I am in favour of your suggestion. I have a commit working locally that implements this, but I'll tidy it up first before committing!

_asyncio_loop_factory.__name__,
hook_factories,
indirect=True,
scope=loop_scope,
)


@contextlib.contextmanager
def _temporary_event_loop_policy(policy: AbstractEventLoopPolicy) -> Iterator[None]:
old_loop_policy = _get_event_loop_policy()
Expand Down Expand Up @@ -798,12 +867,16 @@ def _create_scoped_runner_fixture(scope: _ScopeName) -> Callable:
)
def _scoped_runner(
event_loop_policy,
_asyncio_loop_factory,
request: FixtureRequest,
) -> Iterator[Runner]:
new_loop_policy = event_loop_policy
debug_mode = _get_asyncio_debug(request.config)
with _temporary_event_loop_policy(new_loop_policy):
runner = Runner(debug=debug_mode).__enter__()
runner = Runner(
debug=debug_mode,
loop_factory=_asyncio_loop_factory,
).__enter__()
try:
yield runner
except Exception as e:
Expand All @@ -830,6 +903,11 @@ def _scoped_runner(
)


@pytest.fixture(scope="session")
def _asyncio_loop_factory(request: FixtureRequest) -> LoopFactory | None:
return getattr(request, "param", None)


@pytest.fixture(scope="session", autouse=True)
def event_loop_policy() -> AbstractEventLoopPolicy:
"""Return an instance of the policy used to create asyncio event loops."""
Expand Down
Loading
Loading