write_constraints_file can produce constraints that conflict with toplevel requirements.txt pins

[wjhrdy]

Summary

When the dependency graph contains two versions of the same package — one from a toplevel exact pin and one from a transitive install dependency with a loose specifier — write_constraints_file selects the highest version that satisfies all transitive constraints, even when that differs from the toplevel pin. This produces a constraints.txt that contradicts the toplevel requirements.txt, causing pip to reject the install:

pip install -r requirements.txt -c constraints.txt
# ERROR: conflicting dependencies

This is a practical correctness issue: pip's documented behavior since 20.3 is that "constraints don't override the existing requirements; they simply constrain what versions are visible as input to the resolver." When a constraint conflicts with a requirement, pip refuses to install (pip docs, pip issue #6495). So the output of write_constraints_file must be compatible with the original toplevel pins.

Reproducer

Failing test at https://github.com/wjhrdy/fromager/blob/toplevel-constraint-bug/tests/test_toplevel_local_version.py

Graph structure:

• ROOT → pkg-a==2.0 (toplevel, exact pin)
• ROOT → pkg-b==1.0 (toplevel)
• pkg-b==1.0 → pkg-a==2.1 (install dep via pkg-a>=1.0)

Both pkg-a==2.0 and pkg-a==2.1 exist in the graph. pkg-a==2.0 satisfies the transitive >=1.0 constraint.

Expected constraint: pkg-a==2.0 (the toplevel pin, which satisfies all consumers)
Actual constraint: pkg-a==2.1 (the highest version, overriding the toplevel pin)

$ pytest tests/test_toplevel_local_version.py -v -s

# NOTE: fromager selected pkg-a==2.1 from: ['2.0', '2.1']
pkg-a==2.1

If requirements.txt has pkg-a==2.0 and constraints.txt has pkg-a==2.1, pip rejects the install.

Mechanism

In write_constraints_file (bootstrap.py), the multi-version resolution iterates reversed(sorted(usable_versions.items())) — highest version first — and picks the first version that satisfies all consumers. This "pick highest" strategy works well when there's no toplevel pin, but when the user has an exact toplevel pin that satisfies all transitive constraints, the constraint writer should respect it rather than override it with a different version.

Real-World Impact

This was discovered in the RHAIIS (Red Hat AI Inference Server) downstream build. The toplevel requirement vllm==0.13.0+rhai10 was pinned in requirements.txt, but a transitive dependency (vllm>=0.10.0) was independently resolved to vllm==0.13.0+rhai9. The constraint writer selected rhai9 (which PEP 440 considers higher due to lexicographic local version comparison), and the container build failed:

ERROR: Cannot install vllm==0.13.0+rhai10 because these package versions have conflicting dependencies.
    The user requested vllm==0.13.0+rhai10
    The user requested (constraint) vllm==0.13.0+rhai9

Note on test_write_constraints_file_multiples

The existing test test_write_constraints_file_multiples in test_bootstrap.py asserts the current behavior: toplevel b==0.26.1 is overridden by transitive b==0.26.2. This test may need to be updated if the behavior is changed, or it may reflect an intentionally different design choice — in which case the pip compatibility concern should be documented.

[wjhrdy]

Update: This is not a fromager bug. After further investigation, the behavior is intentional.

The existing test test_write_constraints_file_multiples in test_bootstrap.py asserts the same pattern: when toplevel b==0.26.1 and transitive b==0.26.2 both exist, fromager selects b==0.26.2. The selection loop iterates reversed(sorted(usable_versions.items())), deliberately picking the highest version that satisfies all constraints.

In our real-world case, 0.13.0+rhai9 was selected over 0.13.0+rhai10 because PEP 440 treats rhai10 as a single non-numeric local version segment, compared lexicographically: "rhai9" > "rhai10" (since "9" > "1"). So fromager correctly picked what PEP 440 considers the highest version.

The actual fix is to change our version scheme from +rhaiN to +rhai.N, so PEP 440 splits the local version into ('rhai', 10) and compares the numeric part correctly.

This issue can be closed — apologies for the false report.

[wjhrdy]

Reopening with updated framing. After further investigation, the "pick highest version" behavior is intentional (the existing test_write_constraints_file_multiples asserts it). However, this creates a practical problem when constraints.txt is used alongside requirements.txt via pip install -r requirements.txt -c constraints.txt.

pip's documented behavior since 20.3 (user guide) is:

Constraints don't override the existing requirements; they simply constrain what versions are visible as input to the resolver.

When a constraint conflicts with a requirement, pip refuses to install (pip#6495). There's no PEP governing resolver behavior or constraint file semantics (PEP 440 covers version ordering, PEP 508 covers dependency syntax), but pip's constraint contract is clear: the output must be compatible with the input requirements.

The updated reproducer test uses plain versions (pkg-a==2.0 vs pkg-a==2.1) to isolate this from PEP 440 local version ordering. The toplevel exact pin pkg-a==2.0 satisfies all transitive constraints (pkg-a>=1.0), but the constraint writer picks pkg-a==2.1 instead, which would cause pip to reject the install.

[dhellmann]

We use requirements and constraints a little differently from pip. We do use pins in requirements to ensure specific versions are built. But we support listing multiple versions there, which pip does not. The inputs are meant to guide the resolution process, not necessarily the output process. The output constraints file is expected to give a set of compatible packages. If you want to enforce that the output includes only specific versions, the input constraints should be used.

I could see us declaring this behavior confusing enough to be a bug, but I think it's actually working as intended. Whether we should have intended it that way is up for discussion.

As we move away downstream from using fromager's output constraints files I've been considering removing the generation step from the build command. We would still have a way to generate a constraints file from a graph file, but we wouldn't get one automatically as the result of a build/bootstrap step. This would give us some flexibility in how we produce that file, and we could take in another constraints file, options to choose the highest or lowest version by default, etc. I don't know how useful any of that would be in practice, but for me the point is that it separates the build step from the step of producing that file.