Skip to content

refactor(pathfinder): evaluate replacing spawned_process_runner with dedicated subprocess entrypoints #1771

@cpcloud

Description

@cpcloud

Context

In PR #1768 we switched the CTK canary probe away from multiprocessing.get_context(\"spawn\") and back to a dedicated python -m ... subprocess entrypoint, while keeping cuda.pathfinder._utils.spawned_process_runner in place for the callable-based isolation paths used by tests.

During review, a follow-on idea came up: could cuda_pathfinder use the same fully isolated subprocess pattern for those tests as well, and potentially eliminate spawned_process_runner.py entirely?

Motivation

Right now cuda_pathfinder has two different subprocess-isolation mechanisms:

  • dedicated module subprocesses for the canary probe
  • multiprocessing spawn via spawned_process_runner.py for test helpers / callable execution

This works today, but it means we have two different models to reason about when debugging subprocess behavior, import resolution, loader-state isolation, and script-entrypoint behavior.

Question to investigate

Would it be better to replace the remaining spawned_process_runner.py usage in cuda_pathfinder with the same dedicated-subprocess approach used by the canary probe?

Things to evaluate:

  • whether the test helpers can be expressed cleanly as module/file entrypoints instead of spawned Python callables
  • whether the fully isolated subprocess approach preserves the same loader-state guarantees that motivated spawn in the first place
  • whether this would simplify maintenance by making production and test isolation use one mechanism
  • whether there are places where callable-based spawn is still meaningfully better and should be kept

Scope

This is a follow-on refactor / design issue, not a blocker for #1768.

Metadata

Metadata

Assignees

No one assigned

    Labels

    cuda.pathfinderEverything related to the cuda.pathfinder moduletriageNeeds the team's attention

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions