Implement the branch-hinting proposal (parse + Cranelift cold blocks)

[gfx]

Refs #9463. Supersedes #12483 (closed); this reworks that attempt to address
the review there.

What this does

Parses the metadata.code.branch_hint custom section and uses the hints to mark
cold blocks during Cranelift compilation, behind a new opt-in knob:

• Config::wasm_branch_hinting(bool) — off by default. Also -W branch-hinting
  (CLI) and wasmtime_config_wasm_branch_hinting_set (C API).
• Applies to both br_if and if (the proposal hints both). For a hinted
  branch, the unlikely successor is marked cold via FunctionBuilder::set_cold_block.
• Hints are advisory: when the knob is off the section is ignored exactly as
  today, and when on it never changes execution semantics.

Addressing the #12483 review

• O(n), no per-offset map (cfallin/alexcrichton): the proposal lists hints in
  ascending PC order, so they are consumed with a forward cursor as translation
  visits increasing offsets (FuncEnvironment::take_branch_hint) — O(1)
  amortized, O(n) per function, no HashMap<offset, _>. We rely on the spec's
  ordering rather than re-sorting (validating malformed sections is left to a
  follow-up, see below).
• Config knob, default off (fitzgen): there is no WasmFeatures bit for
  branch hinting (it is a pure custom section with nothing to validate in
  wasmparser), so the knob lives in Tunables and is read by both the parser
  (ModuleEnvironment) and the compiler (FuncEnvironment). Per the proposal
  stability rules it stays off until fuzzed.
• if is handled (a gap in Implement branch hinting proposal support #12483, which only did br_if): all hints in
  tests/spec_testsuite/custom/branch_hint.wast are on if. The if case
  includes the subtlety that the else block is sometimes allocated lazily
  (when params == results); a likely-taken hint there is deferred via
  ControlStackFrame::If { else_is_cold } and applied when Operator::Else
  creates the block.

The hint offset uses builder.srcloc().bits() (module-relative) minus the
function body's start (body.get_binary_reader().original_position()), matching
wasmparser::BranchHint.func_offset.

Config::wasm_branch_hinting only affects cold-block layout, so it is excluded
from the compiled-artifact compatibility check in engine/serialization.rs
(an artifact loads into an engine configured either way).

Codegen evidence

In-tree, tests/disas/branch-hinting-aarch64.wat checks $if_unlikely with the
knob on: the unlikely then-block is hoisted out of line past ret, so the hot
path falls through. tests/disas/branch-hinting-disabled.wat is the same module
without the flag and asserts that nothing is marked cold.

The off-vs-on contrast is clearest on a hot loop — from the objdump A/B in the
dogfood experiment linked below, per iteration the hot path goes from two taken
branches to one:

off — cold inline, hot path branches over it every iteration:
    cbnz x0, <cold>    ; ~always taken, skip the inline cold body
    <cold body>        ; wedged between the hot body and the loop back-edge
    add  x3, #1
    b    <loop>

on — cold hoisted past `ret`, hot path falls through:
    cbz  x0, <cold>    ; ~never taken
    add  x3, #1
    b    <loop>
    ret
    <cold body>        ; out of line

Performance

The transformation is the expected one (hot path tightened, cold path moved out
of line), but I'm not claiming a microbenchmark number here: on a tight loop on
Apple Silicon the wall-time delta is within noise — the eliminated forward
branch is perfectly predicted and the loop fits in L1i, so layout barely
matters. Branch hints pay off under i-cache pressure / large hot code, so the
credible place for a number is /bench_x64 on a representative workload rather
than a microbenchmark. There is no runtime overhead either way: hints only
affect compile-time block layout, and with the knob off codegen is unchanged.

This was dogfooded end-to-end against Wado
output (a language that emits metadata.code.branch_hint): an objdump A/B on a
real Wado component shows the hints take effect. The harness and numbers are
published as a reproducible experiment:
wado-lang/wado#1152.

Testing

• tests/disas/branch-hinting.wat (x86_64, 5 cases): br_if/if × likely/unlikely,
  plus the lazily-allocated-else case — each marks the expected block cold.
• tests/disas/branch-hinting-aarch64.wat (aarch64, test = "compile"): cold
  paths laid out-of-line in real machine code (the optimization is in
  target-independent Cranelift, so it applies on every backend).
• tests/disas/branch-hinting-disabled.wat: the same hinted module with the knob
  off produces no cold blocks (no-regression).
• tests/all/branch_hinting.rs: a hinted module produces identical results with
  the knob enabled, explicitly disabled, and at its default — confirming
  semantic neutrality.

Scope and follow-ups

This is intentionally the codegen + knob slice. Tracked as follow-ups before
flipping the default on:

• Validate the section and reject bad hints (the spec restricts hints to br_if
  and if only). branch_hint.wast exercises both assert_malformed_custom
  (duplicate hint, hint outside a function) and assert_invalid_custom (hint
  whose offset points at a non-branch instruction). Phase 1 ignores misplaced
  hints rather than rejecting them; making these pass needs the validation plus
  assert_{malformed,invalid}_custom support in the wast runner, then ungating
  the test (currently skipped in crates/test-util/src/wast.rs).
• wasm-smith branch-hint emission + fuzzing soak, then enable by default and
  update module_generation_uses_expected_proposals.

The stability matrix is updated accordingly (Tier 3, API/C-API done, Tests/Fuzzed
pending, off by default).

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds opt-in support for the WebAssembly branch-hinting proposal by parsing metadata.code.branch_hint and using it to mark cold blocks during Cranelift compilation, with corresponding user-facing knobs and regression tests.

Changes:

• Introduces a Config::wasm_branch_hinting toggle (plus CLI/C-API plumbing) to enable/disable parsing and use of branch-hint metadata.
• Parses the metadata.code.branch_hint custom section into per-function hint lists and threads them through Cranelift translation to mark cold blocks for if/br_if.
• Adds disassembly and runtime tests (including a “disabled by default” case) plus release/docs updates.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/disas/branch-hinting.wat	New disas test ensuring hints mark the expected cold blocks when enabled.
tests/disas/branch-hinting-disabled.wat	New disas regression test ensuring hints are ignored by default (no cold blocks).
tests/disas/branch-hinting-aarch64.wat	New backend coverage showing hint-driven cold-path layout on aarch64.
tests/all/main.rs	Wires in the new Rust test module.
tests/all/branch_hinting.rs	Adds semantic-neutrality test across enabled/disabled/default configurations.
docs/stability-wasm-proposals.md	Documents branch-hinting status and adds tracking footnote.
crates/wasmtime/src/engine/serialization.rs	Treats branch_hinting as non-compat-affecting for serialized artifacts.
crates/wasmtime/src/config.rs	Adds Config::wasm_branch_hinting public API.
crates/environ/src/tunables.rs	Adds branch_hinting tunable with default false.
crates/environ/src/compile/module_environ.rs	Stores parsed branch hints in ModuleTranslation when enabled.
crates/cranelift/src/translate/stack.rs	Tracks deferred “else is cold” state for lazily-allocated else blocks.
crates/cranelift/src/translate/code_translator.rs	Applies branch hints for if and br_if by marking cold successor blocks.
crates/cranelift/src/func_environ.rs	Adds per-function hint cursoring and take_branch_hint lookup by srcloc.
crates/cranelift/src/compiler.rs	Feeds per-function hints and function-body start offset into FuncEnvironment.
crates/cli-flags/src/lib.rs	Exposes -Wbranch-hinting / wasm proposal flag wiring (opt-in, not in “all”).
crates/c-api/src/config.rs	Adds C API setter to enable/disable branch-hinting.
crates/c-api/include/wasmtime/config.h	Declares the new C API configuration property.
RELEASES.md	Notes the new opt-in branch-hinting support in release notes.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

Label Messager: wasmtime:config

It looks like you are changing Wasmtime's configuration options. Make sure to
complete this check list:

• If you added a new Config method, you wrote extensive documentation for
  it.

  Details

  Our documentation should be of the following form:

  Short, simple summary sentence.
  
  More details. These details can be multiple paragraphs. There should be
  information about not just the method, but its parameters and results as
  well.
  
  Is this method fallible? If so, when can it return an error?
  
  Can this method panic? If so, when does it panic?
  
  # Example
  
  Optional example here.
  

• If you added a new Config method, or modified an existing one, you
  ensured that this configuration is exercised by the fuzz targets.

  Details

  For example, if you expose a new strategy for allocating the next instance
  slot inside the pooling allocator, you should ensure that at least one of our
  fuzz targets exercises that new strategy.

  Often, all that is required of you is to ensure that there is a knob for this
  configuration option in wasmtime_fuzzing::Config (or one
  of its nested structs).

  Rarely, this may require authoring a new fuzz target to specifically test this
  configuration. See our docs on fuzzing for more details.

• If you are enabling a configuration option by default, make sure that it
  has been fuzzed for at least two weeks before turning it on by default.

---

Details

To modify this label's message, edit the .github/label-messager/wasmtime-config.md file.

To add new label messages or remove existing label messages, edit the
.github/label-messager.json configuration file.

Learn more.