docs: update build performance benchmarks (3.4.1)#647
Conversation
Greptile SummaryThis PR is an automated benchmark update for version 3.4.1, appending new measurement rows across all benchmark tables in Confidence Score: 5/5Safe to merge — data is internally consistent, prior review concern is addressed, and the SKILL.md improvement is additive All percentage deltas, per-file averages, raw totals, and 50k extrapolations were spot-checked against the embedded JSON and cross-file references in README.md — every figure is consistent. The engine parity divergence that was flagged in the previous review is now surfaced with a prominent callout note and a tracking issue rather than being silently overwritten. The SKILL.md change is purely additive documentation. No logic bugs, data inconsistencies, or unaddressed concerns remain. No files require special attention Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
CI[CI workflow run #484] -->|produces| RAW[Raw benchmark JSON]
RAW -->|appended to| BBM[BUILD-BENCHMARKS.md\n3.4.1 rows + raw totals]
BBM -->|per-file averages| README[README.md\nperf table updated]
BBM -->|parity gap detected| NOTE[Engine parity note\nNative 10,982 vs WASM 10,956 nodes]
NOTE -->|references| ISS[GitHub Issue #649\ntracking divergence]
SKILL[.claude/skills/release/SKILL.md] -->|new step 5| ROADMAP_CHECK[Roadmap ordering\ncross-check vs ROADMAP.md]
ROADMAP_CHECK -->|runs every release| DRIFT[Prevents silent\nphase-order drift]
Reviews (2): Last reviewed commit: "fix(docs): restore 3.4.1 raw totals lost..." | Re-trigger Greptile |
| | 3.4.1 | native | 2026-03-27 | 473 | 5.7 ↑8% | 11.7 ~ | 23.2 ~ | 44.1 ~ | 57725 ↑5% | | ||
| | 3.4.1 | wasm | 2026-03-27 | 473 | 11.9 ↓2% | 12.2 ↑4% | 23.2 ~ | 44.1 ~ | 42276 ↑5% | |
There was a problem hiding this comment.
Engine parity gap violates repo policy
CLAUDE.md is explicit: "Both engines must produce identical results. If they diverge, the less-accurate engine has a bug — fix it, don't document the gap."
The 3.4.1 entries reveal a divergence across every parity-sensitive metric:
| Metric | Native | WASM | Delta |
|---|---|---|---|
| Nodes (raw) | 10,982 | 10,956 | –26 |
| Edges (raw) | 20,869 | 20,870 | +1 |
| DB size | 26.0 MB | 19.1 MB | +36 % |
The 26-node shortfall means the WASM engine is silently missing ~0.24 % of symbols on this codebase. The DB size spread (57,725 vs 42,276 bytes/file) suggests the discrepancy extends beyond just node count into how the schema is populated.
This divergence was already present in 3.4.0 (23.2 vs 23.1 nodes/file, 44.2 vs 44.4 edges/file), so this PR doesn't introduce it — but committing another version's benchmark without a linked issue means the gap will continue to be silently overwritten each release. Per the repo's own policy, the correct path is to:
- File a tracking issue for the WASM extraction/resolution bug causing the node shortfall.
- Reference that issue in the benchmark notes or a code comment so future benchmark runs surface the outstanding bug rather than burying it.
Context Used: CLAUDE.md (source)
There was a problem hiding this comment.
Addressed. Created tracking issue #649 for the native/WASM node count divergence (10,982 vs 10,956) and DB size gap (26.0 MB vs 19.1 MB).
Added an engine parity note in the "Raw totals (latest)" section of BUILD-BENCHMARKS.md that references #649, so future benchmark runs surface the outstanding bug rather than burying it.
…034137' into benchmark/build-v3.4.1-20260327-034137
Automated build benchmark update for 3.4.1 from workflow run #484.