Fix/narrowing unreachable branch 3349#3698
Conversation
A conditional expression `A if cond else B` kept both branches in its result type even when `cond` is statically decided by prior narrowing — e.g. `1 if x is not None else None` with `x: int` inferred `Literal[1] | None` instead of `Literal[1]`. The `else` can never run, so its type should be dropped; mypy prunes it. Until now only literal `True`/`False` tests were pruned, via the solver-side `as_bool` check. The binding step already forks the ternary and binds the narrowed subject for each branch, and the negated narrow of the dead branch solves to `Never` (`int & None`). We surface those narrow idxs through a range-keyed `ConditionalReachability` binding and, in the solver's `Expr::If`, drop a branch whose narrow solves to `Never`. The check is restricted to identity -against-`None` narrows (`is` / `is not None`): equality narrows can also reach `Never` (negating `== "x"` on a literal), but mypy and pyright keep those branches live, so pruning them would diverge from the conformance bar. Fixes facebook#3349
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Phase 1 dropped a statically-dead branch from a conditional *expression*. The same vacuous-narrowing reachability applies to `if`/`elif`/`else` statements, which merge through a different path (the Phi flow merge), so they kept dead branches: `if x is not None: y = 1 else: y = "s"` with `x: int` inferred `y: Literal['s', 1]`, and a dead *missing* branch such as `if x is None: pass else: y = 1` falsely reported `y` as possibly uninitialized. Each `if`/`elif`/`else` branch now records the `Key::Narrow` idxs of its entry narrowing — the negation of prior branches' tests (where a dead `else`/`elif` lives) plus its own test (where a dead `if`-body lives). These flow through the merge into `BranchInfo.dead_if_any_never` and a per-branch `MissingBranch` record, unifying the existing `NoReturn`-tail termination channel with the new vacuous-narrowing channel. A branch is dropped from the Phi join when it terminates or its narrowing is vacuous. The reachability check is restricted to narrows that are *purely* identity against `None` (`is` / `is not None`): a narrow that only reaches `Never` through a conjoined equality (`x is not None and x == "c"`) keeps its branch live, matching the reference checkers. For uninitialized-variable accounting, the deferral gate combines the new dead-branch signal with the pre-existing NoReturn-count heuristic: the check is deferred to solve time when EVERY missing branch is statically dead, OR when there are at least as many termination-key branches as missing branches. The first disjunct stops a dead `if`-body from making a variable look possibly-uninitialized; keeping the second ensures the change never newly flags a branch the old heuristic already deferred — e.g. an assignment that dominates a `try` body and is observed in `except`/`finally`.
35c04da to
bd4a82b
Compare
|
Diff from mypy_primer, showing the effect of this PR on open source code: pip (https://github.com/pypa/pip)
- ERROR src/pip/_vendor/urllib3/response.py:638:16-42: Expected a type form, got instance of `tuple[*tuple[type[Exception], ...], Never]` [not-a-type]
+ ERROR src/pip/_vendor/urllib3/response.py:638:16-42: Expected a type form, got instance of `tuple[*tuple[type[Exception], ...], Unknown]` [not-a-type]
tornado (https://github.com/tornadoweb/tornado)
- ERROR tornado/options.py:283:25-37: `options_file` may be uninitialized [unbound-name]
spark (https://github.com/apache/spark)
- ERROR python/pyspark/pandas/frame.py:10212:53-10215:26: Cannot set item in `dict[str, list[str]]` [unsupported-operation]
prefect (https://github.com/PrefectHQ/prefect)
- ERROR src/prefect/tasks.py:556:33-40: `CachePolicy` is not assignable to attribute `cache_policy` with type `_None | None` [bad-assignment]
+ ERROR src/prefect/tasks.py:556:33-40: `CachePolicy` is not assignable to attribute `cache_policy` with type `_None` [bad-assignment]
+ ERROR src/prefect/tasks.py:559:33-37: `None` is not assignable to attribute `cache_policy` with type `_None` [bad-assignment]
- ERROR src/prefect/tasks.py:561:73-85: `CacheKeyFnPolicy | CachePolicy | _None | type[NotSet]` is not assignable to attribute `cache_policy` with type `_None | None` [bad-assignment]
+ ERROR src/prefect/tasks.py:561:73-85: `CacheKeyFnPolicy | CachePolicy | _None | type[NotSet]` is not assignable to attribute `cache_policy` with type `_None` [bad-assignment]
ibis (https://github.com/ibis-project/ibis)
- ERROR ibis/common/deferred.py:97:16-74: Returned type `Deferred | str` is not assignable to declared return type `str` [bad-return]
+ ERROR ibis/common/deferred.py:97:16-74: Returned type `Deferred` is not assignable to declared return type `str` [bad-return]
pandas (https://github.com/pandas-dev/pandas)
- ERROR pandas/core/indexes/interval.py:1523:44-59: No matching overload found for function `pandas.core.dtypes.cast.maybe_downcast_numeric` called with arguments: (DatetimeIndex | TimedeltaIndex | ndarray | Unknown, dtype[float64] | dtype[floating] | dtype[integer] | dtype) [no-matching-overload]
+ ERROR pandas/core/indexes/interval.py:1523:22-59: `ExtensionArray | ndarray` is not assignable to variable `breaks` with type `DatetimeIndex | TimedeltaIndex | ndarray` [bad-assignment]
meson (https://github.com/mesonbuild/meson)
- ERROR mesonbuild/cmake/traceparser.py:217:30-219:54: `dict[str, list[str]] | None` is not assignable to attribute `properties` with type `dict[str, list[str]]` [bad-assignment]
PyGithub (https://github.com/PyGithub/PyGithub)
- ERROR github/RequiredPullRequestReviews.py:221:16-102: Returned type `list[Team] | None` is not assignable to declared return type `list[Team]` [bad-return]
- ERROR github/RequiredPullRequestReviews.py:226:16-102: Returned type `list[NamedUser] | None` is not assignable to declared return type `list[NamedUser]` [bad-return]
|
Primer Diff Classification❌ 1 regression(s) | ✅ 4 improvement(s) | ➖ 3 neutral | 8 project(s) total | +6, -10 errors 1 regression(s) across prefect. error kinds:
Detailed analysis❌ Regression (1)prefect (+3, -2)
✅ Improvement (4)tornado (-1)
spark (-1)
The error is about assigning The PR likely improves how pyrefly handles the type inference for
meson (-1)
PyGithub (-2)
➖ Neutral (3)pip (+1, -1)
ibis (+1, -1)
pandas (+1, -1)
Was this helpful? React with 👍 or 👎 Classification by primer-classifier (2 heuristic, 6 LLM) |
Fixes #3349.
Problem
A branch whose reachability is already decided by prior narrowing was kept in the result type, and a dead missing branch falsely triggered "possibly uninitialized":
Approach
Neveris the reachability signal: a branch whose entry narrowing solves toNever(e.g.int & None) cannot run. The narrowed-subject binding is already created and solved, so deciding this is a cache hit, not new work.Reachability is only known at solve time, but which branches merge is fixed at binding time, so the signal is recorded as a key at binding time and evaluated at solve time — the same idiom pyrefly already uses for
NoReturntails. Ternaries andif/elif/elsemerge through two different paths, hence two commits:63daca631— conditional expressions. A range-keyedKey::ConditionalReachabilityside-channel carries each branch's narrow idxs from binding to solving; the inlineExpr::Ifsolver drops the dead side (falling back tounionwhen the channel is absent, e.g. ternaries in type-annotation positions).bd4a82b2d—if/elif/else. Each branch records its entry-narrow idxs (BranchInfo.dead_if_any_never); the existing Phi termination filter now drops a branch that terminates or has a vacuous narrowing.Soundness gate (identity-vs-
Noneonly)Pruning fires only for pure
is None/is not None(NarrowOp::tests_identity_none, every conjunct of anand/ormust qualify). An equality narrow can also reachNeverbut its branch must stay live:assert x is not Nonefeeds this;assert x > 0does not (a comparison onintstaysint).Uninitialized-variable accounting: a combined deferral gate
The dead-branch signal also lets a dead
if-body stop a variable looking possibly-uninitialized. Rather than replace the pre-existingNoReturn-count heuristic (which over-flaggedtry/except/finallyand loops), the deferral gate now combines them:The first disjunct only adds deferral — it can never newly flag a variable — so the dead-branch improvement is gained without regressing the cases the old heuristic already deferred:
Tests
narrow.rs— 9 ternary cases (prune after guard, concrete dead body/else,Anysubject kept, equality kept, mixedand-equality kept, type-annotation position must not panic).flow_branching.rs—if/elif/elsepruning, dead-missing-branch defined, reachable-missing-branch still flagged, plus two regression guards for the try/except and try/finally false positives.scope.rs— one assertion adjusted.Performance
No new solving passes or fixpoint iterations. Added work is bookkeeping (a small
Vecof narrow idxs per branch, one range-keyed binding per ternary, a usually-empty boxed slice per branch). The deadness check reads an already-solved narrow binding and short-circuits on branches with nois Nonenarrow.Validation
mypy_primer: removes false positives (e.g. PyGithubbad-return, sphinx/jax/mesonunbound-name) and introduces nounbound-nameregressions.Known interaction (
prefect, not a regression in this change)In
prefect/tasks.py,cache_policy's type has noNone, socache_policy is Noneis provably false and the ternary'sNonebranch is correctly pruned — sharpening one assignment from_None | Noneto_None. This surfaces a pre-existing attribute-type-inference limitation (pyrefly infersself.cache_policyfrom the first conditional assignment instead of the explicitUnion[…, None]annotation), producing a spuriousbad-assignment. The same lines already errored before this change (as_None | None) and mypy co-reports them. The fix belongs in attribute-type inference and is out of scope here; tracked separately.