Skip to content

Fix/narrowing unreachable branch 3349#3698

Open
mikeleppane wants to merge 2 commits into
facebook:mainfrom
mikeleppane:fix/narrowing-unreachable-branch-3349
Open

Fix/narrowing unreachable branch 3349#3698
mikeleppane wants to merge 2 commits into
facebook:mainfrom
mikeleppane:fix/narrowing-unreachable-branch-3349

Conversation

@mikeleppane
Copy link
Copy Markdown
Contributor

@mikeleppane mikeleppane commented Jun 5, 2026

Fixes #3349.

Problem

A branch whose reachability is already decided by prior narrowing was kept in the result type, and a dead missing branch falsely triggered "possibly uninitialized":

def f(x: int | None) -> None:
    if x is not None:                    # x is `int` here
        y = 1 if x is not None else None # `else None` can never run
        reveal_type(y)                   # was: Literal[1] | None  →  now: Literal[1]

def g(x: int) -> None:
    if x is not None: y = 1
    else:             y = "s"            # dead branch
    reveal_type(y)                       # was: Literal['s', 1]    →  now: Literal[1]

def h(x: int) -> None:
    if x is None: pass                   # dead branch (never runs)
    else:         y = 1
    print(y)                             # was: "y may be uninitialized"  →  now: clean

Approach

Never is the reachability signal: a branch whose entry narrowing solves to Never (e.g. int & None) cannot run. The narrowed-subject binding is already created and solved, so deciding this is a cache hit, not new work.

Reachability is only known at solve time, but which branches merge is fixed at binding time, so the signal is recorded as a key at binding time and evaluated at solve time — the same idiom pyrefly already uses for NoReturn tails. Ternaries and if/elif/else merge through two different paths, hence two commits:

  • 63daca631 — conditional expressions. A range-keyed Key::ConditionalReachability side-channel carries each branch's narrow idxs from binding to solving; the inline Expr::If solver drops the dead side (falling back to union when the channel is absent, e.g. ternaries in type-annotation positions).
  • bd4a82b2dif/elif/else. Each branch records its entry-narrow idxs (BranchInfo.dead_if_any_never); the existing Phi termination filter now drops a branch that terminates or has a vacuous narrowing.

Soundness gate (identity-vs-None only)

Pruning fires only for pure is None / is not None (NarrowOp::tests_identity_none, every conjunct of an and/or must qualify). An equality narrow can also reach Never but its branch must stay live:

1 if (x is not None and x == "c") else 2   # Literal[1, 2] — kept

assert x is not None feeds this; assert x > 0 does not (a comparison on int stays int).

Uninitialized-variable accounting: a combined deferral gate

The dead-branch signal also lets a dead if-body stop a variable looking possibly-uninitialized. Rather than replace the pre-existing NoReturn-count heuristic (which over-flagged try/except/finally and loops), the deferral gate now combines them:

let deferrable =
    all_missing_deferrable                                   // every missing branch is dead, OR
    || n_missing_branches <= n_branches_with_termination_key; // the existing NoReturn heuristic

The first disjunct only adds deferral — it can never newly flag a variable — so the dead-branch improvement is gained without regressing the cases the old heuristic already deferred:

def k(cls, modname) -> str:
    try:
        qualname = get(cls)   # assigned before the only raising call
        analyze(modname)
    except Err:
        pass
    return qualname           # not flagged (matches mypy/pyright)

Tests

  • narrow.rs — 9 ternary cases (prune after guard, concrete dead body/else, Any subject kept, equality kept, mixed and-equality kept, type-annotation position must not panic).
  • flow_branching.rsif/elif/else pruning, dead-missing-branch defined, reachable-missing-branch still flagged, plus two regression guards for the try/except and try/finally false positives.
  • scope.rs — one assertion adjusted.

Performance

No new solving passes or fixpoint iterations. Added work is bookkeeping (a small Vec of narrow idxs per branch, one range-keyed binding per ternary, a usually-empty boxed slice per branch). The deadness check reads an already-solved narrow binding and short-circuits on branches with no is None narrow.

Validation

  • Full unit suite green; format/lint clean; conformance unchanged.
  • Wide mypy_primer: removes false positives (e.g. PyGithub bad-return, sphinx/jax/meson unbound-name) and introduces no unbound-name regressions.

Known interaction (prefect, not a regression in this change)

In prefect/tasks.py, cache_policy's type has no None, so cache_policy is None is provably false and the ternary's None branch is correctly pruned — sharpening one assignment from _None | None to _None. This surfaces a pre-existing attribute-type-inference limitation (pyrefly infers self.cache_policy from the first conditional assignment instead of the explicit Union[…, None] annotation), producing a spurious bad-assignment. The same lines already errored before this change (as _None | None) and mypy co-reports them. The fix belongs in attribute-type inference and is out of scope here; tracked separately.

A conditional expression `A if cond else B` kept both branches in its
result type even when `cond` is statically decided by prior narrowing — e.g.
`1 if x is not None else None` with `x: int` inferred `Literal[1] | None`
instead of `Literal[1]`. The `else` can never run, so its type should be
dropped; mypy prunes it. Until now only literal `True`/`False` tests were
pruned, via the solver-side `as_bool` check.

The binding step already forks the ternary and binds the narrowed subject
for each branch, and the negated narrow of the dead branch solves to `Never`
(`int & None`). We surface those narrow idxs through a range-keyed
`ConditionalReachability` binding and, in the solver's `Expr::If`, drop a
branch whose narrow solves to `Never`. The check is restricted to identity
-against-`None` narrows (`is` / `is not None`): equality narrows can also
reach `Never` (negating `== "x"` on a literal), but mypy and pyright keep
those branches live, so pruning them would diverge from the conformance bar.

Fixes facebook#3349
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@mikeleppane mikeleppane marked this pull request as draft June 5, 2026 08:50
Phase 1 dropped a statically-dead branch from a conditional *expression*.
The same vacuous-narrowing reachability applies to `if`/`elif`/`else`
statements, which merge through a different path (the Phi flow merge), so
they kept dead branches: `if x is not None: y = 1 else: y = "s"` with
`x: int` inferred `y: Literal['s', 1]`, and a dead *missing* branch such as
`if x is None: pass else: y = 1` falsely reported `y` as possibly
uninitialized.

Each `if`/`elif`/`else` branch now records the `Key::Narrow` idxs of its
entry narrowing — the negation of prior branches' tests (where a dead
`else`/`elif` lives) plus its own test (where a dead `if`-body lives). These
flow through the merge into `BranchInfo.dead_if_any_never` and a per-branch
`MissingBranch` record, unifying the existing `NoReturn`-tail termination
channel with the new vacuous-narrowing channel. A branch is dropped from the
Phi join when it terminates or its narrowing is vacuous.

The reachability check is restricted to narrows that are *purely* identity
against `None` (`is` / `is not None`): a narrow that only reaches `Never`
through a conjoined equality (`x is not None and x == "c"`) keeps its branch
live, matching the reference checkers.

For uninitialized-variable accounting, the deferral gate combines the new
dead-branch signal with the pre-existing NoReturn-count heuristic: the check
is deferred to solve time when EVERY missing branch is statically dead, OR
when there are at least as many termination-key branches as missing branches.
The first disjunct stops a dead `if`-body from making a variable look
possibly-uninitialized; keeping the second ensures the change never newly
flags a branch the old heuristic already deferred — e.g. an assignment that
dominates a `try` body and is observed in `except`/`finally`.
@mikeleppane mikeleppane force-pushed the fix/narrowing-unreachable-branch-3349 branch from 35c04da to bd4a82b Compare June 5, 2026 10:40
@github-actions github-actions Bot added size/xl and removed size/xl labels Jun 5, 2026
@mikeleppane mikeleppane marked this pull request as ready for review June 5, 2026 10:46
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jun 5, 2026

Diff from mypy_primer, showing the effect of this PR on open source code:

pip (https://github.com/pypa/pip)
- ERROR src/pip/_vendor/urllib3/response.py:638:16-42: Expected a type form, got instance of `tuple[*tuple[type[Exception], ...], Never]` [not-a-type]
+ ERROR src/pip/_vendor/urllib3/response.py:638:16-42: Expected a type form, got instance of `tuple[*tuple[type[Exception], ...], Unknown]` [not-a-type]

tornado (https://github.com/tornadoweb/tornado)
- ERROR tornado/options.py:283:25-37: `options_file` may be uninitialized [unbound-name]

spark (https://github.com/apache/spark)
- ERROR python/pyspark/pandas/frame.py:10212:53-10215:26: Cannot set item in `dict[str, list[str]]` [unsupported-operation]

prefect (https://github.com/PrefectHQ/prefect)
- ERROR src/prefect/tasks.py:556:33-40: `CachePolicy` is not assignable to attribute `cache_policy` with type `_None | None` [bad-assignment]
+ ERROR src/prefect/tasks.py:556:33-40: `CachePolicy` is not assignable to attribute `cache_policy` with type `_None` [bad-assignment]
+ ERROR src/prefect/tasks.py:559:33-37: `None` is not assignable to attribute `cache_policy` with type `_None` [bad-assignment]
- ERROR src/prefect/tasks.py:561:73-85: `CacheKeyFnPolicy | CachePolicy | _None | type[NotSet]` is not assignable to attribute `cache_policy` with type `_None | None` [bad-assignment]
+ ERROR src/prefect/tasks.py:561:73-85: `CacheKeyFnPolicy | CachePolicy | _None | type[NotSet]` is not assignable to attribute `cache_policy` with type `_None` [bad-assignment]

ibis (https://github.com/ibis-project/ibis)
- ERROR ibis/common/deferred.py:97:16-74: Returned type `Deferred | str` is not assignable to declared return type `str` [bad-return]
+ ERROR ibis/common/deferred.py:97:16-74: Returned type `Deferred` is not assignable to declared return type `str` [bad-return]

pandas (https://github.com/pandas-dev/pandas)
- ERROR pandas/core/indexes/interval.py:1523:44-59: No matching overload found for function `pandas.core.dtypes.cast.maybe_downcast_numeric` called with arguments: (DatetimeIndex | TimedeltaIndex | ndarray | Unknown, dtype[float64] | dtype[floating] | dtype[integer] | dtype) [no-matching-overload]
+ ERROR pandas/core/indexes/interval.py:1523:22-59: `ExtensionArray | ndarray` is not assignable to variable `breaks` with type `DatetimeIndex | TimedeltaIndex | ndarray` [bad-assignment]

meson (https://github.com/mesonbuild/meson)
- ERROR mesonbuild/cmake/traceparser.py:217:30-219:54: `dict[str, list[str]] | None` is not assignable to attribute `properties` with type `dict[str, list[str]]` [bad-assignment]

PyGithub (https://github.com/PyGithub/PyGithub)
- ERROR github/RequiredPullRequestReviews.py:221:16-102: Returned type `list[Team] | None` is not assignable to declared return type `list[Team]` [bad-return]
- ERROR github/RequiredPullRequestReviews.py:226:16-102: Returned type `list[NamedUser] | None` is not assignable to declared return type `list[NamedUser]` [bad-return]

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jun 5, 2026

Primer Diff Classification

❌ 1 regression(s) | ✅ 4 improvement(s) | ➖ 3 neutral | 8 project(s) total | +6, -10 errors

1 regression(s) across prefect. error kinds: bad-assignment. caused by any_narrow_is_never(). 4 improvement(s) across tornado, spark, meson, PyGithub.

Project Verdict Changes Error Kinds Root Cause
pip ➖ Neutral +1, -1 not-a-type
tornado ✅ Improvement -1 unbound-name determine_definition_status()
spark ✅ Improvement -1 unsupported-operation pyrefly/lib/alt/expr.rs
prefect ❌ Regression +3, -2 bad-assignment any_narrow_is_never()
ibis ➖ Neutral +1, -1 bad-return
pandas ➖ Neutral +1, -1 bad-assignment, no-matching-overload any_narrow_is_never()
meson ✅ Improvement -1 bad-assignment pyrefly/lib/alt/expr.rs
PyGithub ✅ Improvement -2 bad-return any_narrow_is_never()
Detailed analysis

❌ Regression (1)

prefect (+3, -2)

This is a minor regression. The PR improves type narrowing by correctly determining that cache_policy cannot be None at line 550 (inside the persist_result is False branch), which eliminates the None from the ternary expression's type. This changes the inferred type of self.cache_policy from _None | None to just _None (where _None represents the type of NO_CACHE). The root cause of all these errors is a pre-existing pyrefly limitation: it infers self.cache_policy's type from the first conditional assignment (line 550) rather than respecting the explicit annotation on line 561 (Union[CachePolicy, type[NotSet], None]). Before the change, 2 false positives existed (lines 556 and 561, with the attribute typed as _None | None). After the change, 3 false positives exist (lines 556, 559, and 561, with the attribute typed as _None). The new error on line 559 (self.cache_policy = None) appears because None is no longer part of the inferred attribute type. All 3 new errors remain false positives — the explicit annotation on line 561 declares the attribute as Union[CachePolicy, type[NotSet], None], and all assignments are compatible with that type. The net effect is +1 false positive (from 2 to 3), making this a minor regression, though the root cause is a pre-existing attribute-type-inference limitation tracked separately. The dead-branch pruning itself is correct.
Attribution: The changes in pyrefly/lib/alt/expr.rs (ternary dead-branch pruning) and pyrefly/lib/alt/solve.rs (any_narrow_is_never() and Phi branch filtering) caused the type of self.cache_policy on line 550 to change from _None | None to _None. Line 550 has self.cache_policy = None if cache_policy is None else NO_CACHE. Since cache_policy has type CachePolicy | type[NotSet] (no None in its type), the cache_policy is None test is provably false, so the PR correctly prunes the None branch of the ternary, sharpening the inferred type to _None (the type of NO_CACHE). This then causes the attribute type to be inferred as _None instead of _None | None, which makes the other assignments (lines 556, 559, 561) produce bad-assignment errors against the narrower inferred type.

✅ Improvement (4)

tornado (-1)

This is a clear improvement. The removed error was a false positive: options_file is always initialized because sys._getframe(0) always returns a valid frame (never None). The else branch at line 281 is dead code since FrameType narrowed against None produces Never. The PR's new dead-branch detection correctly identifies this pattern and stops flagging options_file as possibly uninitialized.
Attribution: The changes in pyrefly/lib/binding/scope.rs (specifically determine_definition_status() and the new all_missing_deferrable logic) and pyrefly/lib/alt/solve.rs (the updated UninitializedCheck handler that now checks any_narrow_is_never for dead branches) are responsible. The else branch after if frame is not None: has an entry narrowing of frame to FrameType & None = Never, which the new any_narrow_is_never() method in solve.rs detects. This causes the missing branch to be marked as statically dead, so the deferred uninitialized check passes at solve time.

spark (-1)

The code at lines 10212-10215 is a list comprehension: [str(value) if value is not None else None for value in column_name_stats_kv[key]]. The value items come from Spark aggregation results stored in column_name_stats_kv[key], which can genuinely be None (null stat values from Spark). The else None branch is NOT dead - it's a real code path that preserves None values.

The error is about assigning list[str | None] to dict[str, list[str]]. The column_name_stats_kv is declared as Dict[str, List[str]] (via defaultdict(list) with type inference), but the list comprehension [str(value) if value is not None else None for value in ...] produces list[str | None] because the else None branch returns None. This creates a type mismatch when assigning back into the dict.

The PR likely improves how pyrefly handles the type inference for column_name_stats_kv or the assignment context, so that the list[str | None] result is compatible with the inferred dict value type, or the dict's value type is more accurately inferred as List[str | None] rather than List[str], eliminating the false positive unsupported-operation error about setting an item in dict[str, list[str]].

Attribution: The change to AnswersSolver in pyrefly/lib/alt/expr.rs (the Expr::If match arm) now checks ConditionalReachability bindings and drops a ternary branch whose narrow solves to Never. Since str & None = Never, the else None branch is pruned, changing the ternary's inferred type from str | None to str, which removes the false positive unsupported-operation error.

meson (-1)

This is a clear improvement. The PR correctly identifies that when tgt.properties has inferred type dict[str, list[str]] (because in CMakeTarget.__init__, the if properties is None: properties = {} guard ensures self.properties is always assigned a dict[str, list[str]]), the else None branch of the ternary on line 217-219 is dead code. The condition tgt.properties is not None is always True given the attribute's inferred type. Previously, pyrefly included the dead branch's None type in the union, producing dict[str, list[str]] | None, which then conflicted with the attribute's inferred type dict[str, list[str]]. The dead-branch pruning correctly narrows the ternary to dict[str, list[str]], eliminating the false positive.
Attribution: The change to AnswersSolver in pyrefly/lib/alt/expr.rs (lines 448-475) adds dead-branch pruning for ternary expressions. When the ConditionalReachability binding shows that one branch's narrow solves to Never (e.g., tgt.properties is not None when properties: dict[str, list[str]] makes the else None branch's narrow dict[str, list[str]] & None = Never), that branch is dropped from the union. This directly removes the false positive bad-assignment error on line 217 of traceparser.py, because the ternary's type is now dict[str, list[str]] instead of dict[str, list[str]] | None.

PyGithub (-2)

Both removed errors were false positives caused by pyrefly including a dead ternary branch in the inferred return type. The property dismissal_restrictions returns DismissalRestrictions (non-optional), so self.dismissal_restrictions is not None is always true. The else None branch is unreachable. Previously pyrefly inferred list[Team] | None / list[NamedUser] | None by including both branches, causing a spurious bad-return. The PR's dead-branch pruning correctly drops the unreachable None branch, resolving the false positive.
Attribution: The change to AnswersSolver in pyrefly/lib/alt/expr.rs (the ternary expression handling around line 445) now checks ConditionalReachability bindings to detect when a branch's narrowing solves to Never. The any_narrow_is_never() method in pyrefly/lib/alt/solve.rs determines that the is not None narrow on DismissalRestrictions (which can't be None) makes the else branch dead. The ConditionalReachability binding created in pyrefly/lib/binding/expr.rs carries the narrow idxs from binding time to solve time, enabling this pruning.

➖ Neutral (3)

pip (+1, -1)

Same errors at same locations with same error kinds — message wording changed, no behavioral impact.

ibis (+1, -1)

Same errors at same locations with same error kinds — message wording changed, no behavioral impact.

pandas (+1, -1)

This is a swap of one false positive for another. The removed no-matching-overload was incorrect (the function should accept these arguments). The new bad-assignment is also incorrect — it's caused by imprecise return type stubs for maybe_downcast_numeric, not a real bug. Neither mypy nor pyright flags either error. Net effect is roughly neutral — one false positive removed, one introduced.
Attribution: The PR's dead-branch pruning in pyrefly/lib/alt/expr.rs and pyrefly/lib/alt/solve.rs changed how the type of breaks is inferred at line 1523. Previously, the overload resolution for maybe_downcast_numeric failed (producing no-matching-overload). Now, with better narrowing, the overload resolves successfully but its return type ExtensionArray | ndarray conflicts with the declared type of breaks. The narrowing improvements in any_narrow_is_never() and the Phi branch pruning changed the inferred type flowing into the call.


Was this helpful? React with 👍 or 👎

Classification by primer-classifier (2 heuristic, 6 LLM)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pyrefly does not eliminate conditional expression branches based on previous narrowing

1 participant