11---
22name : quality-scan
3- description : Cleans up junk files (SCREAMING_TEXT.md, temp files) and performs comprehensive quality scans across codebase to identify critical bugs, logic errors, caching issues, and workflow problems. Spawns specialized agents for targeted analysis and generates prioritized improvement tasks. Use when improving code quality, before releases, or investigating issues.
3+ description : Validates structural consistency, cleans up junk files (SCREAMING_TEXT.md, temp files), and performs comprehensive quality scans across codebase to identify critical bugs, logic errors, caching issues, and workflow problems. Spawns specialized agents for targeted analysis and generates prioritized improvement tasks. Use when improving code quality, before releases, or investigating issues.
44---
55
66# quality-scan
@@ -26,8 +26,9 @@ This is Socket Security's binary tooling manager (BTM) that:
26262 . ** logic** - Algorithm errors, edge cases, type guards, off-by-one errors
27273 . ** cache** - Cache staleness, race conditions, invalidation bugs
28284 . ** workflow** - Build scripts, CI issues, cross-platform compatibility
29- 5 . ** security** - GitHub Actions workflow security (zizmor scanner)
30- 6 . ** documentation** - README accuracy, outdated docs, missing documentation
29+ 5 . ** workflow-optimization** - CI optimization checks (build-required conditions on cached builds)
30+ 6 . ** security** - GitHub Actions workflow security (zizmor scanner)
31+ 7 . ** documentation** - README accuracy, outdated docs, missing documentation, junior developer friendliness
3132
3233** Why Quality Scanning Matters:**
3334- Catches bugs before they reach production
@@ -107,16 +108,15 @@ pnpm run update
107108
108109<validation >
109110** Expected Results:**
110- - Dependencies updated in socket-cli
111+ - Dependencies updated in socket-btm
111112- Report number of packages updated
112113- Continue with scan even if update fails
113114
114115** Track for reporting:**
115116- Packages updated: N
116117- Update status: Success/Failed (with warning)
117118
118- ** Important:** Only update dependencies in the current repository (socket-cli). Do NOT attempt to update sibling repositories (socket-btm, socket-sbom-generator, ultrathink) as this is out of scope and could have unintended side effects.
119- </validation >
119+ ** Important:** Only update dependencies in the current repository (socket-btm). Do NOT attempt to update sibling repositories (socket-sbom-generator, socket-cli, ultrathink) as this is out of scope and could have unintended side effects.</validation >
120120
121121---
122122
@@ -195,7 +195,56 @@ find . -type f -name '*.log' \
195195
196196---
197197
198- ### Phase 4: Determine Scan Scope
198+ ### Phase 4: Structural Validation
199+
200+ <action >
201+ Run automated consistency checker to validate architectural patterns:
202+ </action >
203+
204+ ** Validation Tasks:**
205+
206+ Run the consistency checker to validate monorepo structure:
207+
208+ ``` bash
209+ node scripts/check-consistency.mjs
210+ ```
211+
212+ ** The consistency checker validates:**
213+ 1 . ** Required files** - README.md, package.json existence
214+ 2 . ** Vitest configurations** - Proper mergeConfig usage
215+ 3 . ** Test scripts** - Correct test patterns per package type
216+ 4 . ** Coverage scripts** - Coverage setup where appropriate
217+ 5 . ** External tools** - external-tools.json format validation
218+ 6 . ** Build output structure** - Standard build/{mode}/out/Final/ layout
219+ 7 . ** Package.json structure** - Standard fields and structure
220+ 8 . ** Workspace dependencies** - Proper workspace:* and catalog: usage
221+
222+ <validation >
223+ ** Expected Results:**
224+ - Errors: 0 (any errors should be reported as Critical findings)
225+ - Warnings: 2 or fewer (expected deviations documented in checker)
226+ - Info: Multiple info messages are normal (observations only)
227+
228+ ** If errors found:**
229+ 1 . Report as Critical findings in the final report
230+ 2 . Include file: line references from checker output
231+ 3 . Suggest fixes based on checker recommendations
232+ 4 . Continue with remaining scans
233+
234+ ** If warnings found:**
235+ - Report as Low findings (these are expected deviations)
236+ - Document in final report under "Structural Validation"
237+
238+ ** Track for reporting:**
239+ - Number of packages validated
240+ - Number of errors/warnings/info messages
241+ - Any architectural pattern violations
242+
243+ </validation >
244+
245+ ---
246+
247+ ### Phase 5: Determine Scan Scope
199248
200249<action >
201250Ask user which scans to run:
@@ -206,8 +255,9 @@ Ask user which scans to run:
2062552 . ** logic** - Logic errors (algorithms, edge cases, type guards)
2072563 . ** cache** - Caching issues (staleness, races, invalidation)
2082574 . ** workflow** - Workflow problems (scripts, CI, git hooks)
209- 5 . ** security** - GitHub Actions security (template injection, cache poisoning, etc.)
210- 6 . ** documentation** - Documentation accuracy (README errors, outdated docs)
258+ 5 . ** workflow-optimization** - CI optimization (build-required checks for cached builds)
259+ 6 . ** security** - GitHub Actions security (template injection, cache poisoning, etc.)
260+ 7 . ** documentation** - Documentation accuracy (README errors, outdated docs)
211261
212262** User Interaction:**
213263Use AskUserQuestion tool:
@@ -236,7 +286,7 @@ If user requests non-existent scan type, report error and suggest valid types.
236286
237287---
238288
239- ### Phase 5 : Execute Scans
289+ ### Phase 6 : Execute Scans
240290
241291<action >
242292For each enabled scan type, spawn a specialized agent using Task tool:
@@ -283,59 +333,20 @@ Scan systematically and report all findings. If no issues found, state that expl
283333- Cache scan: reference.md starting at line ~ 200
284334- Workflow scan: reference.md starting at line ~ 300
285335- Security scan: reference.md starting at line ~ 400
286- - Documentation scan: reference.md starting at line ~ 810
336+ - Workflow-optimization scan: reference.md starting at line ~ 860
337+ - Documentation scan: reference.md starting at line ~ 1040
287338
288339<validation >
289- ** Structured Output Validation:**
290-
291- After each agent returns, validate output structure before parsing:
292-
293- ``` bash
294- # 1. Verify agent completed successfully
295- if [ -z " $AGENT_OUTPUT " ]; then
296- echo " ERROR: Agent returned no output"
297- exit 1
298- fi
299-
300- # 2. Check for findings or clean report
301- if ! echo " $AGENT_OUTPUT " | grep -qE ' (File:.*Issue:|No .* issues found|✓ Clean)' ; then
302- echo " WARNING: Agent output missing expected format"
303- echo " Agent may have encountered an error or found no issues"
304- fi
305-
306- # 3. Verify severity levels if findings exist
307- if echo " $AGENT_OUTPUT " | grep -q " File:" ; then
308- if ! echo " $AGENT_OUTPUT " | grep -qE ' Severity: (Critical|High|Medium|Low)' ; then
309- echo " WARNING: Findings missing severity classification"
310- fi
311- fi
312-
313- # 4. Verify fix suggestions if findings exist
314- if echo " $AGENT_OUTPUT " | grep -q " File:" ; then
315- if ! echo " $AGENT_OUTPUT " | grep -q " Fix:" ; then
316- echo " WARNING: Findings missing suggested fixes"
317- fi
318- fi
319- ```
320-
321- ** Manual Verification Checklist:**
322- - [ ] Agent output includes findings OR explicit "No issues found" statement
323- - [ ] All findings include file: line references
324- - [ ] All findings include severity level (Critical/High/Medium/Low)
325- - [ ] All findings include suggested fixes
326- - [ ] Agent output is parseable and structured
327-
328- ** For each scan completion:**
340+ For each scan completion:
329341- Verify agent completed without errors
330- - Extract findings from agent output (or confirm "No issues found")
342+ - Extract findings from agent output
331343- Parse into structured format (file, issue, severity, fix)
332344- Track scan coverage (files analyzed)
333- - Log any validation warnings for debugging
334345</validation >
335346
336347---
337348
338- ### Phase 6 : Aggregate Findings
349+ ### Phase 7 : Aggregate Findings
339350
340351<action >
341352Collect all findings from agents and aggregate:
@@ -374,7 +385,7 @@ interface Finding {
374385
375386---
376387
377- ### Phase 7 : Generate Report
388+ ### Phase 8 : Generate Report
378389
379390<action >
380391Create structured quality report with all findings:
@@ -389,6 +400,34 @@ Create structured quality report with all findings:
389400** Files Scanned:** N
390401** Findings:** N critical, N high, N medium, N low
391402
403+ ## Dependency Updates
404+
405+ ** Repositories Updated:** N/4
406+ - ** socket-btm** : N packages updated
407+ - ** socket-sbom-generator** : N packages updated (or "skipped - not found")
408+ - ** socket-cli** : N packages updated (or "skipped - not found")
409+ - ** ultrathink** : N packages updated (or "skipped - not found")
410+
411+ ** Total Packages Updated:** N across all repositories
412+
413+ ## Structural Validation
414+
415+ ** Consistency Checker Results:**
416+ - Packages validated: 12
417+ - Errors: N (reported as Critical below)
418+ - Warnings: N (reported as Low below)
419+ - Info: N observations
420+
421+ ** Validation Categories:**
422+ ✓ Required files
423+ ✓ Vitest configurations
424+ ✓ Test scripts
425+ ✓ Coverage scripts
426+ ✓ External tools
427+ ✓ Build output structure
428+ ✓ Package.json structure
429+ ✓ Workspace dependencies
430+
392431## Critical Issues (Priority 1) - N found
393432
394433### packages/node-smol-builder/src/patcher.mts:89
@@ -413,6 +452,8 @@ Create structured quality report with all findings:
413452
414453## Scan Coverage
415454
455+ - ** Dependency updates** : N/4 repositories updated (socket-btm, socket-sbom-generator, socket-cli, ultrathink)
456+ - ** Structural validation** : 12 packages validated, 8 architectural patterns checked
416457- ** Critical scan** : N files analyzed in packages/node-smol-builder/, scripts/
417458- ** Logic scan** : N files analyzed (patch logic, build scripts)
418459- ** Cache scan** : N files analyzed (if applicable)
@@ -447,7 +488,7 @@ Create structured quality report with all findings:
447488
448489---
449490
450- ### Phase 8 : Complete
491+ ### Phase 9 : Complete
451492
452493<completion_signal>
453494``` xml
@@ -460,13 +501,27 @@ Report these final metrics to the user:
460501
461502** Quality Scan Complete**
462503========================
504+ ✓ Dependency updates: N/4 repositories updated (N packages)
505+ ✓ Structural validation: N packages validated (N errors, N warnings)
463506✓ Repository cleanup: N junk files removed
464507✓ Scans completed: [ list of scan types]
465508✓ Total findings: N (N critical, N high, N medium, N low)
466509✓ Files scanned: N
467510✓ Report generated: Yes
468511✓ Scan duration: [ calculated from start to end]
469512
513+ ** Dependency Update Summary:**
514+ - socket-btm: N packages updated
515+ - socket-sbom-generator: N packages updated (or "skipped - not found")
516+ - socket-cli: N packages updated (or "skipped - not found")
517+ - ultrathink: N packages updated (or "skipped - not found")
518+
519+ ** Structural Validation Summary:**
520+ - Packages validated: 12
521+ - Consistency errors: N (included in critical findings)
522+ - Consistency warnings: N (included in low findings)
523+ - Architectural patterns checked: 8
524+
470525** Repository Cleanup Summary:**
471526- SCREAMING_TEXT.md files removed: N
472527- Temporary test files removed: N
@@ -506,8 +561,9 @@ See `reference.md` for detailed agent prompts with structured tags:
506561- ** logic-scan** - Off-by-one errors, type guards, edge cases, algorithm correctness
507562- ** cache-scan** - Invalidation, key generation, memory management, concurrency
508563- ** workflow-scan** - Scripts, package.json, git hooks, CI configuration
564+ - ** workflow-optimization-scan** - CI optimization checks (build-required on installation steps with checkpoint caching)
509565- ** security-scan** - GitHub Actions workflow security (runs zizmor scanner)
510- - ** documentation-scan** - README accuracy, outdated examples, incorrect package names, missing documentation
566+ - ** documentation-scan** - README accuracy, outdated examples, incorrect package names, missing documentation, junior developer friendliness (beginner-friendly explanations, troubleshooting, getting started guides)
511567
512568All agent prompts follow Claude best practices with <context >, <instructions >, <pattern >, <output_format>, and <quality_guidelines> tags.
513569
0 commit comments