ci: enhance coverage checks and documentation

Added a new Python script to check line coverage against a specified threshold, integrated into the CI workflow. Updated the test workflow to run all test projects with coverage and enforce a minimum line coverage of 80%. Enhanced the CONTRIBUTING.md to reflect these changes and added a Codecov badge to the README for better visibility of coverage status.

Author: Nick Ficano

.github/workflows/test.yml

@@ -75,20 +75,23 @@ jobs:
       - name: Build
         run: dotnet build ARCP.slnx --configuration Release --no-restore
 
-      - name: Test (unit, with coverage)
+      - name: Test (all test projects, with coverage)
         run: >
-          dotnet test tests/Arcp.UnitTests/Arcp.UnitTests.fsproj
+          dotnet test ARCP.slnx
           --configuration Release
           --no-build
           --verbosity normal
           --collect:"XPlat Code Coverage"
-          --logger "trx;LogFileName=unit-tests.trx"
+          --logger "trx;LogFilePrefix=tests"
           --logger "console;verbosity=normal"
           --results-directory ${{ github.workspace }}/TestResults
 
-      # coverlet.collector writes one coverage.cobertura.xml per test
-      # project under TestResults/<guid>/. Non-blocking so a Codecov
-      # outage cannot break CI.
+      - name: Coverage gate (line coverage >= 80%)
+        run: |
+          python3 scripts/check-coverage.py --threshold 80 --results-dir "${{ github.workspace }}/TestResults"
+
+      # coverlet.collector writes one coverage.cobertura.xml per test project
+      # under TestResults/<guid>/. Non-blocking so a Codecov outage cannot break CI.
       - name: Upload coverage to Codecov
         # codecov/codecov-action v6.0.1
         uses: codecov/codecov-action@e79a6962e0d4c0c17b229090214935d2e33f8354 # v6.0.1

CONTRIBUTING.md

@@ -103,6 +103,8 @@ The full coverage report is regenerated with:
 ```sh
 dotnet test ARCP.slnx --collect:"XPlat Code Coverage" \
   --results-directory TestResults/review-coverage
+python3 scripts/check-coverage.py --threshold 80 \
+  --results-dir TestResults/review-coverage
 reportgenerator \
   -reports:"TestResults/review-coverage/*/coverage.cobertura.xml" \
   -targetdir:"TestResults/coverage-report" \
@@ -111,9 +113,10 @@ reportgenerator \
 
 Install the report tool once with
 `dotnet tool install -g dotnet-reportgenerator-globaltool`. The summary lands
-at `TestResults/coverage-report/Summary.txt`. The target is ≥ 80 % line
-coverage; transport and async-state-machine paths drive most of the
-remaining branch gaps and additions there are welcome.
+at `TestResults/coverage-report/Summary.txt`. CI enforces a minimum **80% line
+coverage** union across all test projects before uploading the reports to
+Codecov; transport and async-state-machine paths drive most of the remaining
+branch gaps and additions there are welcome.
 
 ## Coding standards
 

README.md

@@ -5,6 +5,7 @@
 <p align="center">
   <a href="https://www.nuget.org/packages/Arcp"><img alt="NuGet" src="https://img.shields.io/nuget/v/Arcp.svg"></a>
   <a href="https://github.com/agentruntimecontrolprotocol/fsharp-sdk/actions/workflows/test.yml"><img alt="CI" src="https://github.com/agentruntimecontrolprotocol/fsharp-sdk/actions/workflows/test.yml/badge.svg"></a>
+  <a href="https://codecov.io/gh/agentruntimecontrolprotocol/fsharp-sdk"><img alt="codecov" src="https://codecov.io/gh/agentruntimecontrolprotocol/fsharp-sdk/graph/badge.svg"></a>
   <a href="https://github.com/agentruntimecontrolprotocol/spec/blob/main/docs/draft-arcp-1.1.md"><img alt="ARCP" src="https://img.shields.io/badge/ARCP-v1.1%20draft-blue"></a>
   <a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-Apache--2.0-lightgrey"></a>
 </p>

scripts/check-coverage.py

@@ -0,0 +1,72 @@
+#!/usr/bin/env python3
+"""Compute union line/branch coverage from cobertura reports and fail if line coverage
+is below the requested threshold. Run after `dotnet test ... --collect:"XPlat Code Coverage"`.
+
+Usage:
+  python3 scripts/check-coverage.py --threshold 80 --results-dir TestResults
+"""
+import argparse
+import sys
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+from pathlib import Path
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--threshold", type=float, default=80.0, help="Minimum line-coverage percent.")
+    ap.add_argument("--results-dir", type=Path, default=Path("TestResults"))
+    args = ap.parse_args()
+
+    covered_lines: dict[str, set[int]] = defaultdict(set)
+    valid_lines: dict[str, set[int]] = defaultdict(set)
+    covered_branches: dict[str, set[tuple[int, int]]] = defaultdict(set)
+    valid_branches: dict[str, set[tuple[int, int]]] = defaultdict(set)
+
+    reports = list(args.results_dir.rglob("coverage.cobertura.xml"))
+    if not reports:
+        print(f"No coverage.cobertura.xml found under {args.results_dir}", file=sys.stderr)
+        return 1
+    for cobertura in reports:
+        tree = ET.parse(cobertura)
+        for cls in tree.iter("class"):
+            fname = cls.get("filename", "")
+            for line in cls.iter("line"):
+                ln = int(line.get("number", "0"))
+                hits = int(line.get("hits", "0"))
+                valid_lines[fname].add(ln)
+                if hits > 0:
+                    covered_lines[fname].add(ln)
+                if line.get("branch", "false").lower() == "true":
+                    cc = line.get("condition-coverage", "")
+                    if "(" in cc and "/" in cc:
+                        seg = cc[cc.index("(") + 1:cc.index(")")]
+                        try:
+                            cov, tot = (int(x) for x in seg.split("/"))
+                        except ValueError:
+                            continue
+                        for i in range(tot):
+                            valid_branches[fname].add((ln, i))
+                        for i in range(cov):
+                            covered_branches[fname].add((ln, i))
+
+    total_valid = sum(len(s) for s in valid_lines.values())
+    total_covered = sum(len(covered_lines[f] & valid_lines[f]) for f in valid_lines)
+    bvalid = sum(len(s) for s in valid_branches.values())
+    bcov = sum(len(covered_branches[f] & valid_branches[f]) for f in valid_branches)
+    line_pct = (100 * total_covered / total_valid) if total_valid else 0.0
+    branch_pct = (100 * bcov / bvalid) if bvalid else 0.0
+
+    print(f"Reports merged: {len(reports)}")
+    print(f"Line coverage: {total_covered}/{total_valid} = {line_pct:.2f}%")
+    print(f"Branch coverage: {bcov}/{bvalid} = {branch_pct:.2f}%")
+    print(f"Threshold: {args.threshold:.2f}% (line)")
+
+    if line_pct + 1e-9 < args.threshold:
+        print(f"FAIL: line coverage {line_pct:.2f}% is below threshold {args.threshold:.2f}%", file=sys.stderr)
+        return 1
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())