feat: improve analysis fidelity with doc coverage and semantic gates

Author: baboonzero

README.md

@@ -9,6 +9,7 @@ Outputs include:
 - Mermaid source diagrams (`.mmd`)
 - Rendered SVG and PNG diagrams
 - Confidence, attribution, and quality reports (`meta/*.json`)
+- Documentation coverage report (`meta/coverage_report.json`)
 
 ## Repository Layout
 
@@ -143,9 +144,15 @@ python scripts/analyze.py analyze \
   --output <output_dir> \
   --mode standard \
   --audience nontech \
+  --overview-length medium \
   --enable-web-enrichment true
 ```
 
+Useful optional controls:
+
+- `--include-glob "<pattern>"` (repeatable) to scope analysis to specific paths
+- `--exclude-glob "<pattern>"` (repeatable) to remove generated/irrelevant files
+
 ## Install From GitHub (For Other Developers)
 
 Using Skills CLI:

code-explainer/SKILL.md

@@ -34,13 +34,17 @@ python scripts/analyze.py analyze \
   --output <output_dir> \
   --mode <quick|standard|deep> \
   --audience <nontech|mixed|engineering> \
+  --overview-length <short|medium|long> \
+  --include-glob <pattern> \
+  --exclude-glob <pattern> \
   --enable-web-enrichment <true|false>
 ```
 
 Defaults:
 
 - `mode=standard`
 - `audience=nontech`
+- `overview-length=medium`
 - `enable-web-enrichment=true`
 
 ## Dependencies
@@ -72,12 +76,13 @@ bash ./scripts/install_runtime.sh
 1. Intake and source normalization.
 2. Local index build (files/modules/symbol candidates).
 3. Stack/entrypoint/dependency/flow extraction.
-4. Optional DeepWiki + web enrichment with attribution.
-5. Mermaid generation (Context + Container + flow set).
-6. Mermaid validation.
-7. SVG then PNG rendering.
-8. Overview + deep markdown generation.
-9. Quality gates and confidence report generation.
+4. Documentation ingestion (`coverage_report.json`).
+5. Optional DeepWiki + web enrichment with attribution.
+6. Mermaid generation (Context + Container + flow set).
+7. Mermaid validation.
+8. SVG then PNG rendering.
+9. Overview + deep markdown generation.
+10. Quality gates and confidence report generation.
 
 ## Notes
 

code-explainer/assets/templates/deep_architecture.md.j2

@@ -3,6 +3,10 @@
 Generated: {{generated_at}}  
 Architecture Pattern: **{{architecture_pattern}}**
 
+## Purpose Snapshot
+
+{{plain_summary}}
+
 ## System and Container View
 
 - Context diagram: `../diagrams/svg/c4_context.svg`
@@ -20,11 +24,14 @@ Architecture Pattern: **{{architecture_pattern}}**
 
 {{critical_paths}}
 
+## Documentation Signals Used
+
+{{docs_coverage}}
+
 ## Where To Modify for Common Changes
 
 {{where_to_modify}}
 
 ## Diagram Inventory
 
 {{diagram_links}}
-

code-explainer/assets/templates/deep_flows.md.j2

@@ -6,6 +6,10 @@ Generated: {{generated_at}}
 
 {{request_lifecycle}}
 
+## Primary User Flow (Extracted)
+
+`{{primary_user_flow_summary}}`
+
 ## Flow and Sequence Diagrams
 
 - Request lifecycle: `../diagrams/svg/request_lifecycle_sequence.svg`
@@ -24,4 +28,4 @@ If mode is `deep`, additional assets are also available:
 1. Start with one user flow.
 2. Trace the same path in entrypoint and service layers.
 3. Confirm dependency boundaries before proposing changes.
-
+4. Cross-check assumptions using `meta/coverage_report.json` and `meta/confidence_report.json`.

code-explainer/assets/templates/deep_modules.md.j2

@@ -11,13 +11,17 @@ Generated: {{generated_at}}
 - Modules with larger file counts often represent primary feature or platform surfaces.
 - Use `module_dependency_graph.svg` to validate coupling before changing shared modules.
 - Prioritize low-coupling modules for first onboarding changes.
+- Use doc coverage below to anchor module understanding in existing repository docs.
 
 ## Where To Modify for Feature Work
 
 {{where_to_modify}}
 
+## Doc Coverage Context
+
+{{docs_coverage}}
+
 ## Key Technology Context
 
 - Languages: {{top_languages}}
 - Frameworks: {{top_frameworks}}
-

code-explainer/assets/templates/overview.md.j2

@@ -3,35 +3,50 @@
 Generated: {{generated_at}}  
 Source: `{{source}}`  
 Mode: `{{mode}}`  
-Audience: `{{audience}}`
+Audience: `{{audience}}`  
+Length: `{{overview_length}}`
 
 ## What This System Is
 
-This repository contains the core software system for **{{repo_name}}**.  
+{{plain_summary}}
+
 It appears to follow a **{{architecture_pattern}}** architecture with a stack centered on:
 
 - Languages: {{top_languages}}
 - Frameworks: {{top_frameworks}}
 
 ## Who Uses It and Why
 
-- PMs and designers use this explainer to understand product surfaces and user flows.
-- New engineers use this explainer to identify entry points and module boundaries.
-- Senior engineers use deep docs to evaluate architecture and dependency risk.
+- {{audience_note}}
+- {{mode_note}}
+- {{overview_length_note}}
 
 ## Key Building Blocks (3-7)
 
 {{building_blocks}}
 
+## Documentation Coverage
+
+{{docs_coverage}}
+
+Start with these docs:
+
+{{docs_quick_links}}
+
 ## High-Level Architecture Diagram
 
 ![C4 Context](../diagrams/svg/c4_context.svg)
 
 ## If You Are PM / Designer / New Engineer, Start Here
 
 1. Read this file top-to-bottom once.
-2. Open the user flow image: `../diagrams/svg/primary_user_flow.svg`.
+2. Open `../diagrams/svg/primary_user_flow.svg` and trace the flow: `{{primary_user_flow_summary}}`.
 3. Jump to module ownership and “where to modify” in the deep docs.
+4. If needed, inspect `meta/coverage_report.json` and `meta/confidence_report.json` for caveats.
+
+## External Context (Optional Enrichment)
+
+{{external_context_summary}}
 
 ## Deep Dive Links
 
@@ -40,4 +55,3 @@ It appears to follow a **{{architecture_pattern}}** architecture with a stack ce
 - [Flows Deep Explainer](../deep/FLOWS_DEEP.md)
 - [Dependencies Deep Explainer](../deep/DEPENDENCIES_DEEP.md)
 - [Glossary](../deep/GLOSSARY.md)
-

code-explainer/references/mode-behavior.md

@@ -7,6 +7,7 @@ Goal: Fast orientation with lightweight outputs.
 - Reduced dependency/flow depth.
 - Core docs still generated.
 - Fewer inferred critical paths.
+- Lower documentation ingestion cap.
 
 ## Standard (Default)
 
@@ -15,6 +16,7 @@ Goal: Balanced fidelity and runtime for most repositories.
 - Full required two-tier docs.
 - Mandatory standard diagram set.
 - Context + container + request + dependency views.
+- Balanced doc ingestion and semantic quality checks.
 
 ## Deep
 
@@ -23,4 +25,16 @@ Goal: Maximum fidelity and audit-ready onboarding.
 - Adds advanced diagrams and richer flow analysis.
 - Includes trust boundary and data lineage outputs.
 - Better suited for large or complex repos.
+- Higher doc ingestion cap and longer extracted flow traces.
 
+## Audience Behavior
+
+- `nontech`: plain-language phrasing first, minimal jargon.
+- `mixed`: business-and-technical balance.
+- `engineering`: technical detail and traceability emphasis.
+
+## Overview Length
+
+- `short`: executive skim.
+- `medium`: balanced default.
+- `long`: expanded onboarding context and references.

code-explainer/references/output-contract.md

@@ -23,8 +23,9 @@
 19. `meta/mermaid_validation.json`
 20. `meta/render_report.json`
 21. `meta/enrichment.json`
-22. `meta/docs_generation.json`
-23. `meta/quality_report.json`
+22. `meta/coverage_report.json`
+23. `meta/docs_generation.json`
+24. `meta/quality_report.json`
 
 ## Manifest Schema
 
@@ -40,6 +41,25 @@
 - `entrypoints`
 - `module_count`
 - `diagram_count`
+- `audience`
+- `overview_length`
+- `include_globs[]`
+- `exclude_globs[]`
+- `docs_discovered`
+- `docs_parsed`
+
+## Coverage Schema
+
+`coverage_report.json` contains:
+
+- `generated_at`
+- `mode`
+- `discovered_count`
+- `parsed_count`
+- `skipped_count`
+- `discovered_docs[]`
+- `parsed_docs[]` with `path`, `title`, `summary`, `headings[]`, `line_count`, `size_bytes`, `keywords[]`
+- `skipped_docs[]` with `path`, `reason`
 
 ## Confidence Schema
 
@@ -63,4 +83,3 @@
 - `source_type` (`local`, `web`, or `deepwiki`)
 - `source_uri`
 - `extraction_timestamp`
-