Skip to content

Support wiring of Cython-compiled modules#965

Open
keyz182 wants to merge 1 commit into
ets-labs:developfrom
keyz182:fix/cython-cyfunction-wiring-discovery-on-develop
Open

Support wiring of Cython-compiled modules#965
keyz182 wants to merge 1 commit into
ets-labs:developfrom
keyz182:fix/cython-cyfunction-wiring-discovery-on-develop

Conversation

@keyz182
Copy link
Copy Markdown

@keyz182 keyz182 commented May 18, 2026

Note:

I used Claude Code heavily here. I don't see any policy on your stance on AI use, so if you're against it, I can drop the PR.

That said, this works against our codebase when we run it through cythonize (previously we could only cythonize code that didn't have dependency injection).

Problem

Container.wire() silently skips functions and methods in user modules compiled to .so via Cython. Depends(Provide[Container.x]) markers in those modules fall through unresolved as the literal _Marker sentinel, surfacing later as AttributeError from the handler body.

Failure shape (real-world FastAPI handler):

File "member_service/entrypoints/http/current_identity.py", line 21, in __call__
File "member_service/service/user.py", line 50, in get_user_account_by_email
AttributeError: __aexit__

The uow parameter received the _Marker sentinel because the wiring discovery pass never registered the cyfunction.

Root cause

wiring.py gates on inspect.isfunction / inspect.ismethod, both of which return False for cython_function_or_method instances. Affected call sites in wiring.py:

  • wire() module-member dispatch — elif isfunction(member): (skips compiled module-level functions)
  • _is_method()ismethod(m) or isfunction(m) (skips compiled class methods, including __call__)
  • unwire() traversal — if isfunction(member) + getmembers(member, isfunction) (no-op cleanup for compiled targets)
  • _get_patched() sync/async dispatch — iscoroutinefunction / isasyncgenfunction (Cython < 3 wraps coroutine cyfunctions with the sync patcher)

_cwiring.pyx is unaffected — it operates on the already-built injections dict and never inspects user functions. The blindness is entirely in the discovery pass.

Safety analysis

Verified that dep-injector never mutates user functions:

  • _patch_fn / _patch_method wrap via functools.wraps (writes go to the new pure-Python wrapper, not the cyfunction) and rebind the attribute on the parent module/class via setattr.
  • _fetch_reference_injections only reads inspect.signature(fn), which works on cyfunctions built with embedsignature=True.
  • Zero writes to fn.__defaults__, fn.__code__, fn.__globals__, or fn.__signature__.

Cython's read-only restrictions on those slots are therefore irrelevant.

Change

src/dependency_injector/wiring.py:

  • New private helpers: _is_cyfunction, _is_function_like, _iscoroutinefunction_compat, _isasyncgenfunction_compat.
  • Replace isfunction(member) at the four discovery gates with _is_function_like(member).
  • Replace iscoroutinefunction(fn) / isasyncgenfunction(fn) in _get_patched with the compat helpers, which fall back to __code__.co_flags & CO_COROUTINE / CO_ASYNC_GENERATOR so Cython < 3 async / async-gen cyfunctions are dispatched correctly.
  • Detection scoped to cython_function_or_method only. fused_cython_function templates dispatch per call; wrapping the template would inject before type dispatch — intentionally excluded as potential follow-up work.

Pure-Python wiring behaviour is unchanged. _is_function_like(obj) = isfunction(obj) or _is_cyfunction(obj) only widens the accept set.

Tests

tests/unit/wiring/test_cython.py — 15 tests covering:

  • Cyfunction detection on compiled sync / async / async-gen functions; rejection on pure-Python.
  • _iscoroutinefunction_compat / _isasyncgenfunction_compat on real compiled handlers AND via a synthetic __code__.co_flags callable to exercise the Cython < 3 fallback branches.
  • Runtime injection for sync / async / async-gen / class __call__ shapes.
  • Provider override semantics across the compile boundary.
  • Positive _patched_registry assertion that unwire() empties the injection bindings while preserving reference_injections.
  • Descriptor-type pin on HandlerClass.__call__ so a future Cython release that changes the class-attribute representation surfaces as a test failure rather than silent wiring drop-out.
  • Module-level symbol-presence assertion so an empty / stale .so fails loud at collection time instead of producing a silent 0 collected CI run.

tests/unit/wiring/conftest.py — session-start build of the .pyx fixture via cythonize + build_ext with bare module name + inplace=0 + build_lib pointed at the fixture dir (the .so lands where the test importer expects, not at the repo root that a dotted-or-undotted name with inplace=1 would produce on a clean checkout). Freshness check includes EXT_SUFFIX so a stray .so from a different ABI/Python version is treated as a miss. Build skips silently if Cython / setuptools / a C toolchain is unavailable.

tests/unit/samples/wiringcython/cythonmodule.pyx — fixture exercising the four wiring dispatch shapes with binding=True, embedsignature=True, annotation_typing=False.

tox.inicython>=3,<4 added to [testenv] deps so the default tox invocation builds the fixture. Without it, test_cython.py silently skips.

.gitignore — fixture build artefacts (*.c, *.so, _build/).

docs/wiring.rst — new "Wiring of Cython-compiled modules" section documenting the required cythonize compiler directives (with a callout that annotation_typing defaults to True in Cython 3.x and must be flipped to False for FastAPI handlers using param: str = Header(...) patterns).

Validation

Check Result
dep-injector own test suite + 15 new cython tests 1378 passed, 0 failed, 2 skipped (skips unrelated to this change)
Real-world FastAPI codebase compiled to 51 .so files, run against patched wiring.py 304 passed, 0 cython-specific failures — parity with the equivalent pure-Python image
Same codebase against unpatched wiring.py 289 passed, 15 handler tests fail with the unresolved-sentinel symptom
Behaviour parity vs unpatched wiring.py for pure-Python wiring identical

Out of scope (follow-up)

  • _patch_fn does setattr(module, name, fn) with no read-back verification. Read-only slot / descriptor edge cases could silently no-op. Pre-existing behaviour unchanged by this PR; flagged for a separate hardening pass.
  • fused_cython_function template support. Would require deciding whether to wrap before type dispatch or per-resolved-instance.

Reviewer notes

  • _is_cyfunction uses type(obj).__name__ == "cython_function_or_method" rather than an isinstance check against a Cython-imported type, to avoid making Cython a runtime import requirement of dep-injector itself. The name has been stable across Cython 0.29 → 3.x.
  • The co_flags fallback branches are unreachable from real Cython 3.x cyfunctions (the project pins Cython ≥ 3.1.4). They exist as defensive coverage for downstream users on older Cython. Synthetic tests exercise them.

The wiring discovery pass gated on inspect.isfunction / ismethod, which
return False for cython_function_or_method, silently skipping handlers
compiled to .so. Recognise cyfunctions in discovery and dispatch, fall
back to __code__.co_flags for async-status detection on Cython < 3.

Adds samples/wiringcython/ fixture (sync, async, async-gen, class
__call__) + 15 regression tests. Cython added to [testenv] deps so the
default tox invocation builds the fixture.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant