docs(cookbook): revamp eval-correction-loop + add Mermaid component#660
Open
SuhaniNagpal7 wants to merge 2 commits into
Open
docs(cookbook): revamp eval-correction-loop + add Mermaid component#660SuhaniNagpal7 wants to merge 2 commits into
SuhaniNagpal7 wants to merge 2 commits into
Conversation
added 2 commits
May 21, 2026 13:37
Lazy-loads mermaid@11 from jsdelivr CDN the first time a <Mermaid>
appears on a page. Registered in the auto-import map so cookbook
authors can drop <Mermaid code={`...`} /> into MDX without manual
imports. Theme-aware (re-renders on dark/light toggle) and
deduplicated across multiple diagrams per page via Astro's hoisted
<script> bundling.
- Frontmatter expanded to kebab-case schema (slug, author, products,
frameworks, difficulty, tags, og-image, canonical, last-tested-date,
structured last-tested-with, code-repo-url, page-type).
- Rewrote description for plain-language clarity ("Build a fi.evals
evaluator that matches your team's judgments...").
- Added "What you'll build" section with 5 concrete artifact bullets.
- Expanded "Why this matters" with concrete production-incident lead
(unauthorized refunds) and the four playbook elements: bad outcome,
why standard tooling misses it, which FAGI product helps, metric
that proves the fix.
- Added Mermaid flowchart of the 5-step correction loop.
- Em dashes purged (5 occurrences) per docs style.
- Troubleshooting table gained "Verify" column.
- Replaced "Explore further" reference cards with a 4-item technical
next-steps ladder (CI gate, 80/20 holdout, trace-pipeline promotion,
quarterly recalibration).
- Added real Evaluations dashboard screenshot after Step 4 with
caption explaining how the registered template appears in the list.
This was referenced May 21, 2026
Contributor
|
can you add proper comments in the pr what change you did |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
1. New
MermaidMDX component (src/components/docs/Mermaid.astro)Drop
<Mermaid code={flowchart LR ...} />into any MDX page and it renders. Registered invite-docs-transform.mjsso authors don't need to import it manually. Loadsmermaid@11from jsdelivr lazily (zero cost on pages without diagrams), re-renders on theme toggle via a mutation observer ondata-theme/.dark, and dedupes across multiple diagrams on the same page.2.
eval-correction-loopcookbook revampRestructures
/docs/cookbook/evaluation/eval-correction-loopto match the cookbook playbook:last-tested-with(Python + package versions),time-to-complete,difficulty,code-repo-url,og-image,canonical.Verifycolumn.Why now
The Mermaid component unblocks any future doc/cookbook PR that needs an architecture or flow diagram (PR #641 already imports it for the Imagine page). The eval-correction-loop revamp is the first cookbook to consume it end-to-end, so this PR ships the component and a proof-of-use together.
Test plan
pnpm dev→ visit/docs/cookbook/evaluation/eval-correction-loop. Confirm the 5-step Mermaid flowchart renders, not a raw code block.<Mermaid>. DevTools Network shows nomermaidchunk requested (lazy bundle confirmed).pnpm astro buildpasses; eval-correction-loop page emits adata-mermaidwrapper in the HTML.Scope notes
3 other cookbook files (
semantic-caching,debug-traces-from-ide,docker-compose-quickstart) have related local edits but are not in this PR by design. They'll land in a follow-up once their real dashboard screenshots are captured.