-
Notifications
You must be signed in to change notification settings - Fork 175
Implement event loop factory hook #1373
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Added the ``pytest_asyncio_loop_factories`` hook to parametrize asyncio tests with custom event loop factories. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| ================================================== | ||
| How to use custom event loop factories for tests | ||
| ================================================== | ||
|
|
||
| pytest-asyncio can run asynchronous tests with custom event loop factories by defining a ``pytest_asyncio_loop_factories`` hook in ``conftest.py``. The hook returns the factories to use for the current test item: | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| import asyncio | ||
|
|
||
| import pytest | ||
|
|
||
|
|
||
| class CustomEventLoop(asyncio.SelectorEventLoop): | ||
| pass | ||
|
|
||
|
|
||
| def pytest_asyncio_loop_factories(config, item): | ||
| return [CustomEventLoop] | ||
|
|
||
| When multiple factories are returned, each asynchronous test is run once per factory. Synchronous tests are not parametrized. The configured loop scope still determines how long each event loop instance is kept alive. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There are some test suites that use other plugins for async tests besides pytest-asyncio. Since pytest-trio/pytest-anyio and pytest-asyncio can coexist peacefully, the phrasing "each asynchronous test is run once per factory" made me wonder if this also affects async tests from other plugins. The answer should be obvious, but to clear out any doubt, we could say "each pytest-asyncio test is run once per factory". Or we add another sentence that this doesn't affect tests not managed by pytest-asyncio. |
||
|
|
||
| Factories should be callables without required parameters and should return an ``asyncio.AbstractEventLoop`` instance. The hook must return a non-empty sequence for every asyncio test. | ||
|
|
||
| When multiple ``pytest_asyncio_loop_factories`` implementations are present, pytest-asyncio uses the first non -``None`` result in pytest's normal hook dispatch order. | ||
|
|
||
| To select different factories for specific tests, you can inspect ``item``: | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| import asyncio | ||
|
|
||
| import uvloop | ||
|
|
||
|
|
||
| def pytest_asyncio_loop_factories(config, item): | ||
| if item.get_closest_marker("uvloop"): | ||
| return [uvloop.new_event_loop] | ||
| else: | ||
| return [asyncio.new_event_loop] | ||
|
|
||
| Factory selection can vary per test item, regardless of loop scope. In other words, with ``module``/``package``/``session`` loop scopes you can still choose different factories for different tests by inspecting ``item``. | ||
|
|
||
| .. note:: | ||
|
|
||
| When the hook is defined, async tests are parametrized, so factory names are appended to test IDs. For example, a test ``test_example`` with factory ``CustomEventLoop`` will appear as ``test_example[CustomEventLoop]`` in the test output. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -27,6 +27,7 @@ | |
| Any, | ||
| Literal, | ||
| ParamSpec, | ||
| TypeAlias, | ||
| TypeVar, | ||
| overload, | ||
| ) | ||
|
|
@@ -63,6 +64,7 @@ | |
| _R = TypeVar("_R", bound=Awaitable[Any] | AsyncIterator[Any]) | ||
| _P = ParamSpec("_P") | ||
| FixtureFunction = Callable[_P, _R] | ||
| LoopFactory: TypeAlias = Callable[[], AbstractEventLoop] | ||
|
|
||
|
|
||
| class PytestAsyncioError(Exception): | ||
|
|
@@ -74,6 +76,19 @@ class Mode(str, enum.Enum): | |
| STRICT = "strict" | ||
|
|
||
|
|
||
| hookspec = pluggy.HookspecMarker("pytest") | ||
|
|
||
|
|
||
| class PytestAsyncioSpecs: | ||
| @hookspec | ||
| def pytest_asyncio_loop_factories( | ||
| self, | ||
| config: Config, | ||
| item: Item, | ||
| ) -> Iterable[LoopFactory] | None: | ||
| raise NotImplementedError # pragma: no cover | ||
|
|
||
|
|
||
| ASYNCIO_MODE_HELP = """\ | ||
| 'auto' - for automatically handling all async functions by the plugin | ||
| 'strict' - for autoprocessing disabling (useful if different async frameworks \ | ||
|
|
@@ -83,6 +98,7 @@ class Mode(str, enum.Enum): | |
|
|
||
|
|
||
| def pytest_addoption(parser: Parser, pluginmanager: PytestPluginManager) -> None: | ||
| pluginmanager.add_hookspecs(PytestAsyncioSpecs) | ||
| group = parser.getgroup("asyncio") | ||
| group.addoption( | ||
| "--asyncio-mode", | ||
|
|
@@ -219,6 +235,38 @@ def _get_asyncio_debug(config: Config) -> bool: | |
| return val == "true" | ||
|
|
||
|
|
||
| def _collect_hook_loop_factories( | ||
| config: Config, | ||
| item: Item, | ||
| ) -> tuple[LoopFactory, ...] | None: | ||
| hook_caller = config.hook.pytest_asyncio_loop_factories | ||
| if not hook_caller.get_hookimpls(): | ||
| return None | ||
|
|
||
| results: list[Iterable[LoopFactory] | None] = hook_caller(config=config, item=item) | ||
| msg = "pytest_asyncio_loop_factories must return a non-empty sequence of callables." | ||
| if not results: | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Since I don't think we need to take immediate action here. I merely want to point out that we should keep an eye on whether this has any significant impact on test collection performance. This can make life harder for users with large test suites. |
||
| raise pytest.UsageError(msg) | ||
| result = results[0] | ||
| if result is None or not isinstance(result, Sequence): | ||
| raise pytest.UsageError(msg) | ||
| # Copy into an immutable snapshot so later mutations of the hook's | ||
| # original container do not affect stash state or parametrization. | ||
| factories = tuple(result) | ||
| if not factories or any(not callable(factory) for factory in factories): | ||
| raise pytest.UsageError(msg) | ||
| return factories | ||
|
|
||
|
|
||
| def _get_item_loop_scope(item: Item, config: Config) -> _ScopeName: | ||
| marker = item.get_closest_marker("asyncio") | ||
| default_loop_scope = _get_default_test_loop_scope(config) | ||
| if marker is None: | ||
| return default_loop_scope | ||
| else: | ||
| return _get_marked_loop_scope(marker, default_loop_scope) | ||
|
|
||
|
|
||
| _DEFAULT_FIXTURE_LOOP_SCOPE_UNSET = """\ | ||
| The configuration option "asyncio_default_fixture_loop_scope" is unset. | ||
| The event loop scope for asynchronous fixtures will default to the "fixture" caching \ | ||
|
|
@@ -611,6 +659,27 @@ def pytest_pycollect_makeitem_convert_async_functions_to_subclass( | |
| hook_result.force_result(updated_node_collection) | ||
|
|
||
|
|
||
| @pytest.hookimpl(tryfirst=True) | ||
| def pytest_generate_tests(metafunc: pytest.Metafunc) -> None: | ||
| if _get_asyncio_mode( | ||
| metafunc.config | ||
| ) == Mode.STRICT and not metafunc.definition.get_closest_marker("asyncio"): | ||
| return | ||
| if PytestAsyncioFunction.item_subclass_for(metafunc.definition) is None: | ||
| return | ||
| hook_factories = _collect_hook_loop_factories(metafunc.config, metafunc.definition) | ||
| if hook_factories is None: | ||
| return | ||
| metafunc.fixturenames.append(_asyncio_loop_factory.__name__) | ||
| loop_scope = _get_item_loop_scope(metafunc.definition, metafunc.config) | ||
| metafunc.parametrize( | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Let's say a configured configured pytest_asyncio_loop_factories so that all tests to run with uvloop and SelectorEventLoop. There's one specific test that should only be run with uvloop. This means the user would have to switch to the conftest.py file and find some way to identify the test (e.g. custom marker or item name) to tell pytest-asyncio to use a different set of factories for this test. Ergonomically, it would be preferable for the user to perform this change directly at the level of the test without having to change the hook implementation. This would also improve visibility that this test is an exception. Based on the implementation of _get_item_loop_scope my understanding is that we have access to the asyncio marker here. Could we use the information from the marker to modify the test parametrization? I'm thinking of something like this: Do you think this is feasible and/or reasonable? I'd love to hear your opinion on this @tjkuson .
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @seifertm I can see that being easier. The workflow I had in mind was that people would configure their own marks to achieve this (e.g., designing a The main downsides I can think of to the alternative approach you're suggesting:
I think the benefit to the developer experience outweighs these negatives, however, so I am in favour of your suggestion. I have a commit working locally that implements this, but I'll tidy it up first before committing! |
||
| _asyncio_loop_factory.__name__, | ||
| hook_factories, | ||
| indirect=True, | ||
| scope=loop_scope, | ||
| ) | ||
|
|
||
|
|
||
| @contextlib.contextmanager | ||
| def _temporary_event_loop_policy(policy: AbstractEventLoopPolicy) -> Iterator[None]: | ||
| old_loop_policy = _get_event_loop_policy() | ||
|
|
@@ -798,12 +867,16 @@ def _create_scoped_runner_fixture(scope: _ScopeName) -> Callable: | |
| ) | ||
| def _scoped_runner( | ||
| event_loop_policy, | ||
| _asyncio_loop_factory, | ||
| request: FixtureRequest, | ||
| ) -> Iterator[Runner]: | ||
| new_loop_policy = event_loop_policy | ||
| debug_mode = _get_asyncio_debug(request.config) | ||
| with _temporary_event_loop_policy(new_loop_policy): | ||
| runner = Runner(debug=debug_mode).__enter__() | ||
| runner = Runner( | ||
| debug=debug_mode, | ||
| loop_factory=_asyncio_loop_factory, | ||
| ).__enter__() | ||
| try: | ||
| yield runner | ||
| except Exception as e: | ||
|
|
@@ -830,6 +903,11 @@ def _scoped_runner( | |
| ) | ||
|
|
||
|
|
||
| @pytest.fixture(scope="session") | ||
| def _asyncio_loop_factory(request: FixtureRequest) -> LoopFactory | None: | ||
| return getattr(request, "param", None) | ||
|
|
||
|
|
||
| @pytest.fixture(scope="session", autouse=True) | ||
| def event_loop_policy() -> AbstractEventLoopPolicy: | ||
| """Return an instance of the policy used to create asyncio event loops.""" | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The content of this document is great. I just think it should be split over several documents to be consistent with the existing structutre.
Pytest-asyncio roughly follows the Diataxis documentation framework. I suggest to stop the How-to for custom loop factories right here at line 20, move 25–38 into a dedicated how-to guide, and pack the rest of the information into the reference section of the docs.