From daa5178b5f82b5bca76fb13ab2566ef6e44cd9b4 Mon Sep 17 00:00:00 2001 From: Aaron Brethorst Date: Mon, 25 May 2026 13:42:56 -0700 Subject: [PATCH 01/10] docs: design spec for JSON-for-UI output mode --- .../specs/2026-05-25-json-ui-output-design.md | 250 ++++++++++++++++++ 1 file changed, 250 insertions(+) create mode 100644 docs/superpowers/specs/2026-05-25-json-ui-output-design.md diff --git a/docs/superpowers/specs/2026-05-25-json-ui-output-design.md b/docs/superpowers/specs/2026-05-25-json-ui-output-design.md new file mode 100644 index 0000000..f09b88d --- /dev/null +++ b/docs/superpowers/specs/2026-05-25-json-ui-output-design.md @@ -0,0 +1,250 @@ +# JSON-for-UI Output Mode — Design + +**Date:** 2026-05-25 +**Status:** Approved (design + both sign-off items); pending spec review + +## Purpose + +Add a JSON output mode whose shape is built for **UI visualization** on the +consumer side, and ship a **JSON Schema** file so consumers know exactly what to +expect. The output carries *all the same data* the text report does, but +restructured into metadata + summary + grouped results instead of a flat list. + +The driving use case is the Render **one-off job** pattern (see +`api-key-service`): the job prints a single well-formed JSON object to **stdout**, +the caller reads it from the job output, and the exit code signals pass/fail. +There is no database recording in this project — stdout is the channel. + +## Decisions (locked with the user) + +1. **Replace the existing flat `--json`.** The new structured document becomes + the one and only JSON output. The previous behavior (marshalling + `validator.Report` directly as `{ "results": [...] }`) is removed. +2. **Grouped by section.** Top level is `meta` + `summary` + `groups`, where + `groups` is the `server` group followed by one group per data source. +3. **Rich meta + input echo.** `meta` echoes `obaServerURL` and per-data-source + labels + feed URLs + `agencyMapping`. **The `apiKey` is never echoed.** +4. **Error-as-JSON on stdout.** When `--json` is requested and config load or the + run fails, a JSON error object is printed to stdout (not only stderr), exit 2. +5. **Schema-conformance test** with a test-only dependency + (`github.com/santhosh-tekuri/jsonschema/v6`) asserts generated output + validates against the committed schema, preventing schema rot. + +## Architecture + +Rendering is a presentation concern, so the transform lives in the **`report`** +package (which already imports `validator`; adding `config` introduces no import +cycle — `config` depends on neither). + +- A **pure** builder `BuildDocument(rep validator.Report, cfg config.Config, now + time.Time) Document` produces the view model. Purity (an injected `now`) keeps + tests deterministic, consistent with the repo's determinism convention. +- `WriteJSON(w io.Writer, rep validator.Report, cfg config.Config) error` calls + `BuildDocument` with `time.Now().UTC()` and encodes it indented. +- A small `ErrorDocument` + `WriteErrorJSON(w, msg, apiKey)` covers the error + contract. + +`WriteText` is unchanged. `main.go`'s `--json` branch switches to the new +`WriteJSON` signature and gains the error-JSON path. + +### Flow + +``` +config.Load ──err──▶ (--json? WriteErrorJSON : text) ; exit 2 + │ ok +validator.Run ──err──▶ (--json? WriteErrorJSON : text) ; exit 2 + │ ok + Report ──▶ (--json? WriteJSON=BuildDocument : WriteText) ; exit rep.ExitCode() +``` + +## Output shape (success) + +```jsonc +{ + "schemaVersion": "1.0", + "meta": { + "generatedAt": "2026-05-25T17:04:00Z", // RFC3339, UTC + "obaServerURL": "https://api.pugetsound.onebusaway.org", + "dataSources": [ + { + "id": "dataSource[0]", // joins to groups[].id + "index": 0, + "staticGtfsFeedURL": "https://.../gtfs.zip", + "vehiclePositionsURL": "https://.../vp.pb", + "tripUpdatesURL": "https://.../tu.pb", // omitted when not configured + "serviceAlertsURL": "https://.../alerts.pb", // omitted when not configured + "agencyMapping": { "KCM": "1" } // omitted when empty + } + ] + }, + "summary": { + "verdict": "FAIL", // PASS | FAIL (mirrors Report.Worst()) + "exitCode": 1, // mirrors Report.ExitCode() + "total": 9, + "counts": { "pass": 6, "warn": 1, "fail": 2, "skip": 0 } + }, + "groups": [ + { + "id": "server", + "label": "Server", + "counts": { "pass": 2, "warn": 0, "fail": 0, "skip": 0 }, + "results": [ + { + "check": "basic-endpoints/current-time", + "category": "basic-endpoints", + "step": "current-time", + "status": "PASS", + "message": "OK" + } + ] + }, + { + "id": "dataSource[0]", + "label": "Data source 0", + "counts": { "pass": 4, "warn": 1, "fail": 2, "skip": 0 }, + "results": [ + { + "check": "vehicle-positions-sampling/trip-for-vehicle", + "category": "vehicle-positions-sampling", + "step": "trip-for-vehicle", + "status": "FAIL", + "message": "...", + "details": { "vehicleId": "1_1234" } // omitted when nil + } + ] + } + ] +} +``` + +### Field rules + +- **`status`** values stay uppercase (`PASS`/`WARN`/`FAIL`/`SKIP`) to match + `Status.String()` and the prior JSON. **`counts`** keys are lowercase + (`pass`/`warn`/`fail`/`skip`) and always present (zeros included). +- **`category` / `step`** are split from `check` on the **first** `/`. A check + with no `/` (e.g. `agency-union`) yields `category = check`, `step` omitted. + This gives the UI a stable grouping key without re-parsing the `check` string. +- **Grouping** is deterministic: `server` group first (all results with empty + `Source`), then one group per data source in **config order**, keyed by the + source label (`dataSource[N]`). A result whose `Source` matches no known data + source is emitted in a trailing group keyed by that source (sorted) so no data + is ever dropped. +- **`label`** is human-friendly: `"Server"`, `"Data source N"`. +- **`verdict`** is `FAIL` iff `Report.Worst() == Fail`, else `PASS` (matches the + text report). `exitCode` mirrors `Report.ExitCode()` (0 or 1). + +### Security + +`apiKey` is never placed in the document. As defense in depth, every echoed URL +in `meta` is passed through a redactor that replaces the apiKey substring with +`REDACTED` if it ever appears (mirrors the existing `redact(err, key)` rule in +the validator). `Details` values pass through unchanged — checks are already +responsible for redacting them at the source. + +## Output shape (error) + +```json +{ "schemaVersion": "1.0", "error": "obaServerURL is required" } +``` + +Emitted to stdout when `--json` is set and either `config.Load` or +`validator.Run` returns an error; the process exits 2. The message is passed +through the apiKey redactor before printing. + +## JSON Schema file + +`schema/oba-validator-report.schema.json` — JSON Schema **draft 2020-12** with +`$id`, `title`, property `description`s, and `required` arrays. Top level is +`oneOf: [reportDocument, errorDocument]`, both requiring `schemaVersion`, so a +consumer can validate either variant against one file. `additionalProperties` +is `false` on the closed objects (`summary`, `counts`, result items) to catch +drift; `details` is left open (`additionalProperties: true`). + +## View-model types (in `report`) + +```go +type Document struct { + SchemaVersion string `json:"schemaVersion"` + Meta Meta `json:"meta"` + Summary Summary `json:"summary"` + Groups []Group `json:"groups"` +} +type Meta struct { + GeneratedAt string `json:"generatedAt"` + OBAServerURL string `json:"obaServerURL"` + DataSources []MetaSource `json:"dataSources"` +} +type MetaSource struct { + ID string `json:"id"` + Index int `json:"index"` + StaticGtfsFeedURL string `json:"staticGtfsFeedURL,omitempty"` + VehiclePositionsURL string `json:"vehiclePositionsURL,omitempty"` + TripUpdatesURL string `json:"tripUpdatesURL,omitempty"` + ServiceAlertsURL string `json:"serviceAlertsURL,omitempty"` + AgencyMapping map[string]string `json:"agencyMapping,omitempty"` +} +type Summary struct { + Verdict string `json:"verdict"` + ExitCode int `json:"exitCode"` + Total int `json:"total"` + Counts Counts `json:"counts"` +} +type Counts struct { + Pass int `json:"pass"` + Warn int `json:"warn"` + Fail int `json:"fail"` + Skip int `json:"skip"` +} +type Group struct { + ID string `json:"id"` + Label string `json:"label"` + Counts Counts `json:"counts"` + Results []Item `json:"results"` +} +type Item struct { + Check string `json:"check"` + Category string `json:"category"` + Step string `json:"step,omitempty"` + Status string `json:"status"` + Message string `json:"message"` + Details map[string]any `json:"details,omitempty"` +} +type ErrorDocument struct { + SchemaVersion string `json:"schemaVersion"` + Error string `json:"error"` +} +``` + +## Testing (TDD red → green) + +`report` package (no network, table/`httptest` style per repo convention): + +- `BuildDocument`: server vs data-source grouping and ordering; `category`/`step` + split (incl. no-slash case); per-group and summary `counts`; `verdict` and + `exitCode`; `generatedAt` RFC3339 formatting via injected `now`; `meta` echo + of URLs + `agencyMapping`; `omitempty` on unconfigured URLs. +- **apiKey never present:** assert the full marshalled output contains neither + the apiKey nor a URL with the apiKey embedded (URL-redaction test). +- `WriteJSON` produces valid, indented JSON with the expected top-level keys. +- `WriteErrorJSON` produces `{schemaVersion, error}` with the message redacted. +- **Schema conformance:** marshal a representative success `Document` and an + `ErrorDocument`, then validate both against + `schema/oba-validator-report.schema.json` using `santhosh-tekuri/jsonschema/v6`. + Also assert a deliberately-malformed document fails validation (guards that + the schema is actually constraining). + +`cmd` package (existing `httptest` harness style): + +- `--json` against a stubbed run yields top-level `meta`/`summary`/`groups`. +- Config error under `--json` prints `{"error": ...}` to stdout and exits 2. + +The existing flat-output `TestWriteJSON` in `report/report_test.go` is rewritten +to assert the new structure. + +## Out of scope (YAGNI) + +- No database / result-table recording (unlike `api-key-service`); stdout only. +- No new CLI flag — `--json` is repurposed, not added alongside. +- No per-check schema beyond `details: object` (details are check-specific and + intentionally open). From 9610307abb52e2f2e1c4dc892a80aa13262f9102 Mon Sep 17 00:00:00 2001 From: Aaron Brethorst Date: Mon, 25 May 2026 13:46:36 -0700 Subject: [PATCH 02/10] docs: implementation plan for JSON-for-UI output mode --- .../plans/2026-05-25-json-ui-output-mode.md | 1051 +++++++++++++++++ 1 file changed, 1051 insertions(+) create mode 100644 docs/superpowers/plans/2026-05-25-json-ui-output-mode.md diff --git a/docs/superpowers/plans/2026-05-25-json-ui-output-mode.md b/docs/superpowers/plans/2026-05-25-json-ui-output-mode.md new file mode 100644 index 0000000..c976293 --- /dev/null +++ b/docs/superpowers/plans/2026-05-25-json-ui-output-mode.md @@ -0,0 +1,1051 @@ +# JSON-for-UI Output Mode Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Replace the flat `--json` output with a structured, UI-oriented JSON document (meta + summary + grouped results), and ship a JSON Schema file describing it. + +**Architecture:** A pure `BuildDocument(report, config, now)` transform in the `report` package converts the engine's `validator.Report` into a view model; `WriteJSON` encodes it. Errors during a `--json` run are emitted as a JSON `ErrorDocument` to stdout (the Render one-off-job convention). A committed JSON Schema (draft 2020-12) describes both variants, guarded by a conformance test. + +**Tech Stack:** Go 1.25, `encoding/json`, `net/http/httptest` for tests, `github.com/santhosh-tekuri/jsonschema/v6` (test-only) for schema conformance. + +--- + +## File Structure + +- **Create** `report/document.go` — view-model types, `SchemaVersion`, `BuildDocument` + helpers (`splitCheck`, `redactString`, `Counts.add`, `buildMeta`/`buildGroup`/`buildSummary`). +- **Create** `report/document_test.go` — tests for the helpers and `BuildDocument`; shared test fixtures (`sampleReport`, `sampleConfig`, `fixedTime`). +- **Create** `report/schema_test.go` — schema-conformance tests. +- **Create** `schema/oba-validator-report.schema.json` — the published JSON Schema. +- **Modify** `report/report.go` — change `WriteJSON` signature, add `WriteErrorJSON`; `WriteText` unchanged. +- **Modify** `report/report_test.go` — rewrite `TestWriteJSON` to the new shape; drop its local `sampleReport` (moved to `document_test.go`). +- **Modify** `cmd/oba-validator/main.go` — pass `cfg` to `WriteJSON`; emit error JSON on failure under `--json`. +- **Modify** `cmd/oba-validator/main_test.go` — add `--json` shape test and config-error test. +- **Modify** `go.mod` / `go.sum` — add the test-only dependency. +- **Modify** `README.md`, `CLAUDE.md`, and the spec status line — document the new output. + +Naming contract used across tasks (define once, reuse exactly): +`Document{SchemaVersion, Meta, Summary, Groups}`, `Meta{GeneratedAt, OBAServerURL, DataSources}`, `MetaSource{ID, Index, StaticGtfsFeedURL, VehiclePositionsURL, TripUpdatesURL, ServiceAlertsURL, AgencyMapping}`, `Summary{Verdict, ExitCode, Total, Counts}`, `Counts{Pass, Warn, Fail, Skip}`, `Group{ID, Label, Counts, Results}`, `Item{Check, Category, Step, Status, Message, Details}`, `ErrorDocument{SchemaVersion, Error}`. Functions: `BuildDocument(rep validator.Report, cfg config.Config, now time.Time) Document`, `WriteJSON(w io.Writer, rep validator.Report, cfg config.Config) error`, `WriteErrorJSON(w io.Writer, msg, apiKey string) error`. Const: `SchemaVersion = "1.0"`. + +--- + +## Task 1: View-model types + pure helpers + +**Files:** +- Create: `report/document.go` +- Test: `report/document_test.go` + +- [ ] **Step 1: Write the failing test** + +Create `report/document_test.go`: + +```go +package report + +import ( + "testing" +) + +func TestSplitCheck(t *testing.T) { + cases := []struct { + in string + category string + step string + }{ + {"basic-endpoints/current-time", "basic-endpoints", "current-time"}, + {"vehicle-positions-sampling/trip-for-vehicle", "vehicle-positions-sampling", "trip-for-vehicle"}, + {"agency-union", "agency-union", ""}, + {"a/b/c", "a", "b/c"}, + } + for _, c := range cases { + cat, step := splitCheck(c.in) + if cat != c.category || step != c.step { + t.Errorf("splitCheck(%q) = (%q,%q) want (%q,%q)", c.in, cat, step, c.category, c.step) + } + } +} + +func TestRedactString(t *testing.T) { + if got := redactString("https://x/?key=SEKRET", "SEKRET"); got != "https://x/?key=***" { + t.Errorf("redactString did not redact: %q", got) + } + if got := redactString("no-key-here", "SEKRET"); got != "no-key-here" { + t.Errorf("redactString altered non-matching string: %q", got) + } + if got := redactString("anything", ""); got != "anything" { + t.Errorf("empty apiKey should be a no-op: %q", got) + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `go test ./report/ -run 'TestSplitCheck|TestRedactString' -v` +Expected: FAIL — compile error, `undefined: splitCheck`, `undefined: redactString`. + +- [ ] **Step 3: Write minimal implementation** + +Create `report/document.go`: + +```go +package report + +import "strings" + +// SchemaVersion is the version of the JSON document shape emitted by WriteJSON. +// It appears as the top-level "schemaVersion" field of every document. +const SchemaVersion = "1.0" + +// Document is the UI-oriented JSON output of a successful validation run. +type Document struct { + SchemaVersion string `json:"schemaVersion"` + Meta Meta `json:"meta"` + Summary Summary `json:"summary"` + Groups []Group `json:"groups"` +} + +// Meta echoes the run inputs (never the apiKey) so a UI can show what was checked. +type Meta struct { + GeneratedAt string `json:"generatedAt"` // RFC3339, UTC + OBAServerURL string `json:"obaServerURL"` + DataSources []MetaSource `json:"dataSources"` +} + +// MetaSource echoes one configured data source. ID joins to Group.ID. +type MetaSource struct { + ID string `json:"id"` + Index int `json:"index"` + StaticGtfsFeedURL string `json:"staticGtfsFeedURL,omitempty"` + VehiclePositionsURL string `json:"vehiclePositionsURL,omitempty"` + TripUpdatesURL string `json:"tripUpdatesURL,omitempty"` + ServiceAlertsURL string `json:"serviceAlertsURL,omitempty"` + AgencyMapping map[string]string `json:"agencyMapping,omitempty"` +} + +// Summary is the run-wide verdict and tallies. +type Summary struct { + Verdict string `json:"verdict"` // PASS | FAIL + ExitCode int `json:"exitCode"` + Total int `json:"total"` + Counts Counts `json:"counts"` +} + +// Counts tallies results by status. Keys are lowercase; values always present. +type Counts struct { + Pass int `json:"pass"` + Warn int `json:"warn"` + Fail int `json:"fail"` + Skip int `json:"skip"` +} + +// add increments the tally for an uppercase status string (PASS/WARN/FAIL/SKIP). +func (c *Counts) add(status string) { + switch status { + case "PASS": + c.Pass++ + case "WARN": + c.Warn++ + case "FAIL": + c.Fail++ + case "SKIP": + c.Skip++ + } +} + +// Group is one section of results: the server, or one data source. +type Group struct { + ID string `json:"id"` + Label string `json:"label"` + Counts Counts `json:"counts"` + Results []Item `json:"results"` +} + +// Item is one check result, with check name pre-split into category/step. +type Item struct { + Check string `json:"check"` + Category string `json:"category"` + Step string `json:"step,omitempty"` + Status string `json:"status"` + Message string `json:"message"` + Details map[string]any `json:"details,omitempty"` +} + +// ErrorDocument is emitted to stdout when a --json run fails before producing a report. +type ErrorDocument struct { + SchemaVersion string `json:"schemaVersion"` + Error string `json:"error"` +} + +// splitCheck splits a check name on the first '/' into category and step. +// A name without '/' yields (name, ""). +func splitCheck(check string) (category, step string) { + if i := strings.IndexByte(check, '/'); i >= 0 { + return check[:i], check[i+1:] + } + return check, "" +} + +// redactString replaces the apiKey substring with "*** " (matching the +// validator's redact convention) so a secret never reaches output. A no-op +// when apiKey is empty. +func redactString(s, apiKey string) string { + if apiKey == "" { + return s + } + return strings.ReplaceAll(s, apiKey, "***") +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `go test ./report/ -run 'TestSplitCheck|TestRedactString' -v` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add report/document.go report/document_test.go +git commit -m "feat(report): view-model types and pure helpers for JSON document" +``` + +--- + +## Task 2: BuildDocument transform + +**Files:** +- Modify: `report/document.go` +- Test: `report/document_test.go` + +- [ ] **Step 1: Write the failing test** + +Append to `report/document_test.go` (add imports `encoding/json`, `strings`, `time`, `github.com/onebusaway/oba-validator/config`, `github.com/onebusaway/oba-validator/validator`): + +```go +func fixedTime() time.Time { return time.Date(2026, 5, 25, 17, 4, 0, 0, time.UTC) } + +func sampleReport() validator.Report { + return validator.Report{Results: []validator.Result{ + {Check: "basic-endpoints/current-time", Status: validator.Pass, Message: "OK"}, + {Check: "agency-union", Status: validator.Pass, Message: "all agencies present"}, + {Check: "vehicle-positions-sampling/trip-for-vehicle", Source: "dataSource[0]", Status: validator.Fail, Message: "missing", Details: map[string]any{"vehicleId": "1_1234"}}, + {Check: "freshness", Source: "dataSource[0]", Status: validator.Warn, Message: "empty feed"}, + }} +} + +func sampleConfig() config.Config { + return config.Config{ + OBAServerURL: "https://oba.example.org", + APIKey: "secret-key", + DataSources: []config.DataSource{{ + StaticGtfsFeedURL: "https://feeds.example.org/gtfs.zip", + VehiclePositionsURL: "https://feeds.example.org/vp.pb", + AgencyMapping: map[string]string{"KCM": "1"}, + }}, + } +} + +func TestBuildDocument_GroupingAndOrder(t *testing.T) { + doc := BuildDocument(sampleReport(), sampleConfig(), fixedTime()) + if len(doc.Groups) != 2 { + t.Fatalf("groups=%d want 2", len(doc.Groups)) + } + if doc.Groups[0].ID != "server" || doc.Groups[0].Label != "Server" { + t.Errorf("group[0]=%+v want server/Server", doc.Groups[0]) + } + if doc.Groups[1].ID != "dataSource[0]" || doc.Groups[1].Label != "Data source 0" { + t.Errorf("group[1]=%+v want dataSource[0]/Data source 0", doc.Groups[1]) + } + if len(doc.Groups[0].Results) != 2 || len(doc.Groups[1].Results) != 2 { + t.Errorf("group sizes = %d,%d want 2,2", len(doc.Groups[0].Results), len(doc.Groups[1].Results)) + } +} + +func TestBuildDocument_CategoryStep(t *testing.T) { + doc := BuildDocument(sampleReport(), sampleConfig(), fixedTime()) + got := doc.Groups[0].Results + if got[0].Category != "basic-endpoints" || got[0].Step != "current-time" { + t.Errorf("result[0] cat/step = %q/%q", got[0].Category, got[0].Step) + } + if got[1].Category != "agency-union" || got[1].Step != "" { + t.Errorf("result[1] cat/step = %q/%q want agency-union/empty", got[1].Category, got[1].Step) + } +} + +func TestBuildDocument_CountsVerdictExit(t *testing.T) { + doc := BuildDocument(sampleReport(), sampleConfig(), fixedTime()) + if doc.Groups[0].Counts != (Counts{Pass: 2}) { + t.Errorf("server counts=%+v", doc.Groups[0].Counts) + } + if doc.Groups[1].Counts != (Counts{Fail: 1, Warn: 1}) { + t.Errorf("ds counts=%+v", doc.Groups[1].Counts) + } + if doc.Summary.Counts != (Counts{Pass: 2, Warn: 1, Fail: 1}) { + t.Errorf("summary counts=%+v", doc.Summary.Counts) + } + if doc.Summary.Total != 4 { + t.Errorf("total=%d want 4", doc.Summary.Total) + } + if doc.Summary.Verdict != "FAIL" || doc.Summary.ExitCode != 1 { + t.Errorf("verdict/exit = %q/%d want FAIL/1", doc.Summary.Verdict, doc.Summary.ExitCode) + } + if doc.SchemaVersion != SchemaVersion { + t.Errorf("schemaVersion=%q", doc.SchemaVersion) + } +} + +func TestBuildDocument_MetaEcho(t *testing.T) { + doc := BuildDocument(sampleReport(), sampleConfig(), fixedTime()) + if doc.Meta.GeneratedAt != "2026-05-25T17:04:00Z" { + t.Errorf("generatedAt=%q", doc.Meta.GeneratedAt) + } + if doc.Meta.OBAServerURL != "https://oba.example.org" { + t.Errorf("obaServerURL=%q", doc.Meta.OBAServerURL) + } + if len(doc.Meta.DataSources) != 1 { + t.Fatalf("meta dataSources=%d want 1", len(doc.Meta.DataSources)) + } + ms := doc.Meta.DataSources[0] + if ms.ID != "dataSource[0]" || ms.Index != 0 || ms.AgencyMapping["KCM"] != "1" { + t.Errorf("metaSource=%+v", ms) + } + // tripUpdatesURL/serviceAlertsURL were not configured -> omitted from JSON. + b, _ := json.Marshal(doc) + if strings.Contains(string(b), "tripUpdatesURL") || strings.Contains(string(b), "serviceAlertsURL") { + t.Errorf("unconfigured URLs should be omitted:\n%s", b) + } +} + +func TestBuildDocument_RedactsAPIKey(t *testing.T) { + cfg := sampleConfig() + cfg.APIKey = "SEKRET" + cfg.OBAServerURL = "https://oba.example.org/?key=SEKRET" + b, _ := json.Marshal(BuildDocument(sampleReport(), cfg, fixedTime())) + if strings.Contains(string(b), "SEKRET") { + t.Errorf("apiKey leaked into output:\n%s", b) + } + if !strings.Contains(string(b), "***") { + t.Errorf("expected redaction marker in output:\n%s", b) + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `go test ./report/ -run TestBuildDocument -v` +Expected: FAIL — `undefined: BuildDocument`. + +- [ ] **Step 3: Write minimal implementation** + +Append to `report/document.go` (add imports `fmt`, `sort`, `time`, `github.com/onebusaway/oba-validator/config`, `github.com/onebusaway/oba-validator/validator`; keep `strings`): + +```go +// BuildDocument transforms a validation report and its config into the +// UI-oriented Document. It is pure: pass time.Now().UTC() for now in production +// and a fixed time in tests. The apiKey is never echoed; URLs are redacted. +func BuildDocument(rep validator.Report, cfg config.Config, now time.Time) Document { + bySource := map[string][]validator.Result{} + for _, r := range rep.Results { + bySource[r.Source] = append(bySource[r.Source], r) + } + + groups := []Group{buildGroup("server", "Server", bySource[""])} + delete(bySource, "") + for i := range cfg.DataSources { + id := fmt.Sprintf("dataSource[%d]", i) + groups = append(groups, buildGroup(id, fmt.Sprintf("Data source %d", i), bySource[id])) + delete(bySource, id) + } + // Any result with an unrecognized source is emitted in a trailing group + // (sorted) so no data is ever dropped. + leftover := make([]string, 0, len(bySource)) + for k := range bySource { + leftover = append(leftover, k) + } + sort.Strings(leftover) + for _, k := range leftover { + groups = append(groups, buildGroup(k, k, bySource[k])) + } + + return Document{ + SchemaVersion: SchemaVersion, + Meta: buildMeta(cfg, now), + Summary: buildSummary(rep, groups), + Groups: groups, + } +} + +func buildGroup(id, label string, results []validator.Result) Group { + g := Group{ID: id, Label: label, Results: []Item{}} + for _, r := range results { + cat, step := splitCheck(r.Check) + status := r.Status.String() + g.Results = append(g.Results, Item{ + Check: r.Check, + Category: cat, + Step: step, + Status: status, + Message: r.Message, + Details: r.Details, + }) + g.Counts.add(status) + } + return g +} + +func buildMeta(cfg config.Config, now time.Time) Meta { + m := Meta{ + GeneratedAt: now.UTC().Format(time.RFC3339), + OBAServerURL: redactString(cfg.OBAServerURL, cfg.APIKey), + DataSources: make([]MetaSource, 0, len(cfg.DataSources)), + } + for i, ds := range cfg.DataSources { + m.DataSources = append(m.DataSources, MetaSource{ + ID: fmt.Sprintf("dataSource[%d]", i), + Index: i, + StaticGtfsFeedURL: redactString(ds.StaticGtfsFeedURL, cfg.APIKey), + VehiclePositionsURL: redactString(ds.VehiclePositionsURL, cfg.APIKey), + TripUpdatesURL: redactString(ds.TripUpdatesURL, cfg.APIKey), + ServiceAlertsURL: redactString(ds.ServiceAlertsURL, cfg.APIKey), + AgencyMapping: ds.AgencyMapping, + }) + } + return m +} + +func buildSummary(rep validator.Report, groups []Group) Summary { + var total Counts + for _, g := range groups { + total.Pass += g.Counts.Pass + total.Warn += g.Counts.Warn + total.Fail += g.Counts.Fail + total.Skip += g.Counts.Skip + } + verdict := "PASS" + if rep.Worst() == validator.Fail { + verdict = "FAIL" + } + return Summary{ + Verdict: verdict, + ExitCode: rep.ExitCode(), + Total: total.Pass + total.Warn + total.Fail + total.Skip, + Counts: total, + } +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `go test ./report/ -run TestBuildDocument -v` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add report/document.go report/document_test.go +git commit -m "feat(report): BuildDocument transform from Report to UI document" +``` + +--- + +## Task 3: WriteJSON (new shape) + WriteErrorJSON + +**Files:** +- Modify: `report/report.go` +- Modify: `report/report_test.go` +- Modify: `cmd/oba-validator/main.go:84` (call-site only, to keep the build green) + +- [ ] **Step 1: Write the failing test** + +Replace the body of `report/report_test.go` with (note: `sampleReport` now lives in `document_test.go`, so it is removed here): + +```go +package report + +import ( + "bytes" + "encoding/json" + "strings" + "testing" + + "github.com/onebusaway/oba-validator/validator" +) + +func TestWriteJSON(t *testing.T) { + var buf bytes.Buffer + if err := WriteJSON(&buf, sampleReport(), sampleConfig()); err != nil { + t.Fatal(err) + } + var doc Document + if err := json.Unmarshal(buf.Bytes(), &doc); err != nil { + t.Fatalf("output not a Document: %v\n%s", err, buf.String()) + } + if doc.SchemaVersion != SchemaVersion { + t.Errorf("schemaVersion=%q", doc.SchemaVersion) + } + if len(doc.Groups) != 2 || doc.Summary.Verdict != "FAIL" { + t.Errorf("unexpected document: %+v", doc.Summary) + } + if !strings.Contains(buf.String(), "\n ") { + t.Error("expected indented JSON") + } +} + +func TestWriteErrorJSON(t *testing.T) { + var buf bytes.Buffer + if err := WriteErrorJSON(&buf, "boom with SEKRET", "SEKRET"); err != nil { + t.Fatal(err) + } + var ed ErrorDocument + if err := json.Unmarshal(buf.Bytes(), &ed); err != nil { + t.Fatalf("output not an ErrorDocument: %v\n%s", err, buf.String()) + } + if ed.SchemaVersion != SchemaVersion { + t.Errorf("schemaVersion=%q", ed.SchemaVersion) + } + if strings.Contains(ed.Error, "SEKRET") || !strings.Contains(ed.Error, "***") { + t.Errorf("error not redacted: %q", ed.Error) + } +} + +func TestWriteText(t *testing.T) { + var buf bytes.Buffer + if err := WriteText(&buf, sampleReport()); err != nil { + t.Fatal(err) + } + out := buf.String() + if !strings.Contains(out, "✓") || !strings.Contains(out, "✗") { + t.Errorf("missing glyphs:\n%s", out) + } + if !strings.Contains(out, "FAIL") { + t.Errorf("missing summary:\n%s", out) + } +} + +func TestWriteTextSummaryLine(t *testing.T) { + var buf bytes.Buffer + if err := WriteText(&buf, sampleReport()); err != nil { + t.Fatal(err) + } + if !strings.Contains(buf.String(), "FAIL (4 checks, 1 failed, 1 warnings)") { + t.Errorf("summary line wrong:\n%s", buf.String()) + } +} + +var _ = validator.Pass // keep validator import even if unused above +``` + +Note the `TestWriteTextSummaryLine` expectation changed to `(4 checks, 1 failed, 1 warnings)` to match the new `sampleReport`. Remove the `var _ = validator.Pass` line if `validator` ends up referenced elsewhere in the file; it exists only to avoid an unused-import error. + +- [ ] **Step 2: Run test to verify it fails** + +Run: `go test ./report/ -run 'TestWriteJSON|TestWriteErrorJSON' -v` +Expected: FAIL — `WriteJSON` arg count mismatch / `undefined: WriteErrorJSON`. + +- [ ] **Step 3: Write minimal implementation** + +In `report/report.go`, replace `WriteJSON` and add `WriteErrorJSON`. New file body: + +```go +// Package report renders a validator.Report as JSON or human-readable text. +package report + +import ( + "encoding/json" + "fmt" + "io" + "time" + + "github.com/onebusaway/oba-validator/config" + "github.com/onebusaway/oba-validator/validator" +) + +// WriteJSON writes the report as an indented, UI-oriented JSON Document. +func WriteJSON(w io.Writer, rep validator.Report, cfg config.Config) error { + enc := json.NewEncoder(w) + enc.SetIndent("", " ") + return enc.Encode(BuildDocument(rep, cfg, time.Now().UTC())) +} + +// WriteErrorJSON writes an indented ErrorDocument to w, redacting apiKey from msg. +func WriteErrorJSON(w io.Writer, msg, apiKey string) error { + enc := json.NewEncoder(w) + enc.SetIndent("", " ") + return enc.Encode(ErrorDocument{SchemaVersion: SchemaVersion, Error: redactString(msg, apiKey)}) +} + +// WriteText writes a human-readable, grouped report with a summary line. +func WriteText(w io.Writer, rep validator.Report) error { + var fails, warns int + for _, r := range rep.Results { + group := r.Source + if group == "" { + group = "server" + } + if _, err := fmt.Fprintf(w, "%s [%s] %s — %s\n", r.Status.Glyph(), group, r.Check, r.Message); err != nil { + return err + } + switch r.Status { + case validator.Fail: + fails++ + case validator.Warn: + warns++ + } + } + verdict := "PASS" + if rep.Worst() == validator.Fail { + verdict = "FAIL" + } + _, err := fmt.Fprintf(w, "\n%s (%d checks, %d failed, %d warnings)\n", verdict, len(rep.Results), fails, warns) + return err +} +``` + +Then update the call site in `cmd/oba-validator/main.go` (currently line 84) from: + +```go + werr = report.WriteJSON(stdout, rep) +``` + +to: + +```go + werr = report.WriteJSON(stdout, rep, cfg) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `go build ./... && go test ./report/ -v` +Expected: build OK; all `report` tests PASS. + +- [ ] **Step 5: Commit** + +```bash +git add report/report.go report/report_test.go cmd/oba-validator/main.go +git commit -m "feat(report): WriteJSON emits UI document; add WriteErrorJSON" +``` + +--- + +## Task 4: Wire CLI error-JSON path + CLI tests + +**Files:** +- Modify: `cmd/oba-validator/main.go` +- Test: `cmd/oba-validator/main_test.go` + +- [ ] **Step 1: Write the failing test** + +Append to `cmd/oba-validator/main_test.go` (add imports `encoding/json`, `net/http`, `net/http/httptest`, `strings`): + +```go +func TestRunJSONConfigErrorEmitsErrorJSON(t *testing.T) { + var stdout, stderr bytes.Buffer + code := run([]string{"oba-validator", "--json", `{"dataSources":[]}`}, &stdout, &stderr) + if code != 2 { + t.Fatalf("exit=%d want 2", code) + } + var ed struct { + SchemaVersion string `json:"schemaVersion"` + Error string `json:"error"` + } + if err := json.Unmarshal(stdout.Bytes(), &ed); err != nil { + t.Fatalf("stdout not JSON: %v\n%s", err, stdout.String()) + } + if ed.Error == "" || ed.SchemaVersion == "" { + t.Errorf("missing fields in error doc: %s", stdout.String()) + } +} + +func TestRunJSONOutputShape(t *testing.T) { + obaSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + switch { + case strings.Contains(r.URL.Path, "current-time"): + w.Write([]byte(`{"data":{"entry":{"time":1716000000000}}}`)) + case strings.Contains(r.URL.Path, "agencies-with-coverage"): + w.Write([]byte(`{"data":{"list":[],"references":{"agencies":[]}}}`)) + default: + w.Write([]byte(`{"data":{"list":[],"entry":{"arrivalsAndDepartures":[]}}}`)) + } + })) + defer obaSrv.Close() + feedSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Write([]byte{}) // empty payload -> prep error recorded, run still completes + })) + defer feedSrv.Close() + + cfg := `{"obaServerURL":"` + obaSrv.URL + `","apiKey":"test","dataSources":[{"staticGtfsFeedURL":"` + feedSrv.URL + `/gtfs.zip"}]}` + var stdout, stderr bytes.Buffer + run([]string{"oba-validator", "--json", "--no-cache", cfg}, &stdout, &stderr) + + var doc map[string]json.RawMessage + if err := json.Unmarshal(stdout.Bytes(), &doc); err != nil { + t.Fatalf("stdout not JSON: %v\n%s", err, stdout.String()) + } + for _, k := range []string{"schemaVersion", "meta", "summary", "groups"} { + if _, ok := doc[k]; !ok { + t.Errorf("missing key %q in output:\n%s", k, stdout.String()) + } + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `go test ./cmd/... -run TestRunJSON -v` +Expected: FAIL — `TestRunJSONConfigErrorEmitsErrorJSON` gets text on stdout (or empty), not JSON. + +- [ ] **Step 3: Write minimal implementation** + +In `cmd/oba-validator/main.go`, replace the config-error and run-error blocks (currently lines ~69–80) so that, under `--json`, errors are emitted as JSON to stdout. The relevant section of `run` becomes: + +```go + cfg, err := config.Load(fs.Arg(0)) + if err != nil { + if o.jsonOut { + if werr := report.WriteErrorJSON(stdout, err.Error(), os.Getenv("ONEBUSAWAY_API_KEY")); werr != nil { + fmt.Fprintln(stderr, "output error:", werr) + } + } else { + fmt.Fprintln(stderr, "config error:", err) + } + return 2 + } + applyOverrides(&cfg, o) + + rep, err := validator.Run(context.Background(), cfg) + if err != nil { + if o.jsonOut { + if werr := report.WriteErrorJSON(stdout, err.Error(), cfg.APIKey); werr != nil { + fmt.Fprintln(stderr, "output error:", werr) + } + } else { + fmt.Fprintln(stderr, "run error:", err) + } + return 2 + } +``` + +(`os` is already imported in `main.go`.) + +- [ ] **Step 4: Run test to verify it passes** + +Run: `go test ./cmd/... -v` +Expected: all PASS. + +- [ ] **Step 5: Commit** + +```bash +git add cmd/oba-validator/main.go cmd/oba-validator/main_test.go +git commit -m "feat(cli): emit JSON error document on failure under --json" +``` + +--- + +## Task 5: JSON Schema file + conformance test + +**Files:** +- Create: `schema/oba-validator-report.schema.json` +- Create: `report/schema_test.go` +- Modify: `go.mod`, `go.sum` + +- [ ] **Step 1: Add the test-only dependency** + +Run: + +```bash +go get github.com/santhosh-tekuri/jsonschema/v6 +``` + +Expected: `go.mod`/`go.sum` updated with the `jsonschema/v6` module. + +- [ ] **Step 2: Write the failing test** + +Create `report/schema_test.go`: + +```go +package report + +import ( + "encoding/json" + "os" + "path/filepath" + "testing" + + "github.com/santhosh-tekuri/jsonschema/v6" +) + +func compileSchema(t *testing.T) *jsonschema.Schema { + t.Helper() + path := filepath.Join("..", "schema", "oba-validator-report.schema.json") + data, err := os.ReadFile(path) + if err != nil { + t.Fatalf("read schema: %v", err) + } + var doc any + if err := json.Unmarshal(data, &doc); err != nil { + t.Fatalf("schema is not valid JSON: %v", err) + } + c := jsonschema.NewCompiler() + if err := c.AddResource("report.json", doc); err != nil { + t.Fatalf("add resource: %v", err) + } + sch, err := c.Compile("report.json") + if err != nil { + t.Fatalf("compile schema: %v", err) + } + return sch +} + +func validateAgainst(t *testing.T, sch *jsonschema.Schema, v any) error { + t.Helper() + b, err := json.Marshal(v) + if err != nil { + t.Fatal(err) + } + var inst any + if err := json.Unmarshal(b, &inst); err != nil { + t.Fatal(err) + } + return sch.Validate(inst) +} + +func TestSchema_SuccessDocumentConforms(t *testing.T) { + sch := compileSchema(t) + if err := validateAgainst(t, sch, BuildDocument(sampleReport(), sampleConfig(), fixedTime())); err != nil { + t.Errorf("success document failed schema:\n%v", err) + } +} + +func TestSchema_ErrorDocumentConforms(t *testing.T) { + sch := compileSchema(t) + ed := ErrorDocument{SchemaVersion: SchemaVersion, Error: "boom"} + if err := validateAgainst(t, sch, ed); err != nil { + t.Errorf("error document failed schema:\n%v", err) + } +} + +func TestSchema_RejectsMalformed(t *testing.T) { + sch := compileSchema(t) + // Has schemaVersion but is neither a valid report (missing meta/summary/groups) + // nor a valid error (no "error"); oneOf must match zero variants. + bad := map[string]any{"schemaVersion": "1.0", "summary": map[string]any{}} + if err := validateAgainst(t, sch, bad); err == nil { + t.Error("expected malformed document to fail schema validation") + } +} +``` + +- [ ] **Step 3: Run test to verify it fails** + +Run: `go test ./report/ -run TestSchema -v` +Expected: FAIL — schema file does not exist (`read schema: ... no such file`). + +- [ ] **Step 4: Write the schema** + +Create `schema/oba-validator-report.schema.json`: + +```json +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://github.com/onebusaway/oba-validator/schema/oba-validator-report.schema.json", + "title": "OBA Validator Output", + "description": "Output of `oba-validator --json`: either a validation report document or an error document. Both carry a top-level schemaVersion.", + "oneOf": [ + { "$ref": "#/$defs/reportDocument" }, + { "$ref": "#/$defs/errorDocument" } + ], + "$defs": { + "status": { + "type": "string", + "enum": ["PASS", "WARN", "FAIL", "SKIP"] + }, + "counts": { + "type": "object", + "description": "Result tallies by status.", + "additionalProperties": false, + "required": ["pass", "warn", "fail", "skip"], + "properties": { + "pass": { "type": "integer", "minimum": 0 }, + "warn": { "type": "integer", "minimum": 0 }, + "fail": { "type": "integer", "minimum": 0 }, + "skip": { "type": "integer", "minimum": 0 } + } + }, + "item": { + "type": "object", + "description": "One check result. category/step are the check name split on the first '/'.", + "additionalProperties": false, + "required": ["check", "category", "status", "message"], + "properties": { + "check": { "type": "string" }, + "category": { "type": "string" }, + "step": { "type": "string" }, + "status": { "$ref": "#/$defs/status" }, + "message": { "type": "string" }, + "details": { "type": "object", "additionalProperties": true } + } + }, + "group": { + "type": "object", + "description": "A section of results: the server, or one data source.", + "additionalProperties": false, + "required": ["id", "label", "counts", "results"], + "properties": { + "id": { "type": "string", "description": "'server' or 'dataSource[N]'; joins to meta.dataSources[].id." }, + "label": { "type": "string" }, + "counts": { "$ref": "#/$defs/counts" }, + "results": { "type": "array", "items": { "$ref": "#/$defs/item" } } + } + }, + "metaSource": { + "type": "object", + "description": "Echo of one configured data source. The apiKey is never present.", + "additionalProperties": false, + "required": ["id", "index"], + "properties": { + "id": { "type": "string" }, + "index": { "type": "integer", "minimum": 0 }, + "staticGtfsFeedURL": { "type": "string" }, + "vehiclePositionsURL": { "type": "string" }, + "tripUpdatesURL": { "type": "string" }, + "serviceAlertsURL": { "type": "string" }, + "agencyMapping": { "type": "object", "additionalProperties": { "type": "string" } } + } + }, + "meta": { + "type": "object", + "additionalProperties": false, + "required": ["generatedAt", "obaServerURL", "dataSources"], + "properties": { + "generatedAt": { "type": "string", "format": "date-time" }, + "obaServerURL": { "type": "string" }, + "dataSources": { "type": "array", "items": { "$ref": "#/$defs/metaSource" } } + } + }, + "summary": { + "type": "object", + "additionalProperties": false, + "required": ["verdict", "exitCode", "total", "counts"], + "properties": { + "verdict": { "type": "string", "enum": ["PASS", "FAIL"] }, + "exitCode": { "type": "integer", "enum": [0, 1] }, + "total": { "type": "integer", "minimum": 0 }, + "counts": { "$ref": "#/$defs/counts" } + } + }, + "reportDocument": { + "type": "object", + "additionalProperties": false, + "required": ["schemaVersion", "meta", "summary", "groups"], + "properties": { + "schemaVersion": { "type": "string" }, + "meta": { "$ref": "#/$defs/meta" }, + "summary": { "$ref": "#/$defs/summary" }, + "groups": { "type": "array", "items": { "$ref": "#/$defs/group" } } + } + }, + "errorDocument": { + "type": "object", + "additionalProperties": false, + "required": ["schemaVersion", "error"], + "properties": { + "schemaVersion": { "type": "string" }, + "error": { "type": "string" } + } + } + } +} +``` + +- [ ] **Step 5: Run test to verify it passes; tidy modules** + +Run: `go mod tidy && go test ./report/ -run TestSchema -v` +Expected: all `TestSchema*` PASS. + +- [ ] **Step 6: Commit** + +```bash +git add schema/oba-validator-report.schema.json report/schema_test.go go.mod go.sum +git commit -m "feat: publish JSON Schema and conformance test for --json output" +``` + +--- + +## Task 6: Documentation + +**Files:** +- Modify: `README.md` +- Modify: `CLAUDE.md` +- Modify: `docs/superpowers/specs/2026-05-25-json-ui-output-design.md` (status line) + +- [ ] **Step 1: Update README** + +In `README.md`, after the "Library" section (line ~46), add a section: + +```markdown +## JSON output + +`--json` emits a single structured document to stdout, designed for building a UI +visualization. It contains `meta` (run inputs — never the apiKey), `summary` +(verdict + status counts), and `groups` (a `server` group plus one per data +source, each with its results). On failure before a report is produced, a +`{ "schemaVersion, error }` object is emitted to stdout and the process exits 2. + +The full contract is published as a JSON Schema (draft 2020-12) at +[`schema/oba-validator-report.schema.json`](schema/oba-validator-report.schema.json). +This is the recommended format for the Render one-off-job workflow: the job +prints the document to stdout and the caller reads it from the job output. +``` + +Also update the Library snippet line `report.WriteJSON(os.Stdout, rep)` to `report.WriteJSON(os.Stdout, rep, cfg)`. + +- [ ] **Step 2: Update CLAUDE.md** + +In `CLAUDE.md`, in the `report` bullet of the Architecture section, change the description of JSON output from "indented JSON (`WriteJSON`)" to note the new shape: + +```markdown +4. **`report`** — renders a `Report` as grouped text (`WriteText`) or, via + `WriteJSON`, a UI-oriented JSON `Document` (meta + summary + grouped results; + schema at `schema/oba-validator-report.schema.json`). `WriteErrorJSON` emits + the error variant. The `Document` view model is built by the pure + `BuildDocument(report, config, now)` so output is deterministic in tests. +``` + +- [ ] **Step 3: Update spec status** + +In `docs/superpowers/specs/2026-05-25-json-ui-output-design.md`, change the status line to: + +```markdown +**Status:** Implemented +``` + +- [ ] **Step 4: Verify full suite + build** + +Run: `go build ./... && go vet ./... && go test ./...` +Expected: build OK, vet clean, all tests PASS. + +- [ ] **Step 5: Commit** + +```bash +git add README.md CLAUDE.md docs/superpowers/specs/2026-05-25-json-ui-output-design.md +git commit -m "docs: document --json UI output and schema" +``` + +--- + +## Self-Review + +**Spec coverage:** +- Replace flat `--json` → Task 3 (WriteJSON new shape) + Task 3 call-site update. ✓ +- Grouped-by-section shape → Task 2 (BuildDocument grouping). ✓ +- Rich meta + input echo, no apiKey → Task 2 (buildMeta) + redaction test. ✓ +- Error-as-JSON on stdout → Task 4. ✓ +- JSON Schema file → Task 5. ✓ +- Schema-conformance test (test-only dep) → Task 5. ✓ +- category/step split, deterministic ordering, counts/verdict/exit → Task 2. ✓ +- Docs → Task 6. ✓ + +**Placeholder scan:** No TBD/TODO; every code/test step contains complete code and exact run commands. ✓ + +**Type consistency:** `BuildDocument`/`WriteJSON`/`WriteErrorJSON` signatures, the `Document`/`Meta`/`MetaSource`/`Summary`/`Counts`/`Group`/`Item`/`ErrorDocument` field names, and `SchemaVersion` are identical across the file-structure contract, Tasks 1–5, the schema, and the tests. `Counts.add` consumes the uppercase status from `validator.Status.String()`. ✓ + +**Note on existing tests:** `report_test.go`'s `sampleReport` is moved to `document_test.go` and enriched (4 results, 2 sources), so `TestWriteTextSummaryLine`'s expected line is updated to `(4 checks, 1 failed, 1 warnings)` in Task 3. From 87c03b601eb1fdde9657d0f2f2442f20c6d11f92 Mon Sep 17 00:00:00 2001 From: Aaron Brethorst Date: Mon, 25 May 2026 13:47:06 -0700 Subject: [PATCH 03/10] feat(report): view-model types and pure helpers for JSON document --- report/document.go | 106 ++++++++++++++++++++++++++++++++++++++++ report/document_test.go | 36 ++++++++++++++ 2 files changed, 142 insertions(+) create mode 100644 report/document.go create mode 100644 report/document_test.go diff --git a/report/document.go b/report/document.go new file mode 100644 index 0000000..f87b4dd --- /dev/null +++ b/report/document.go @@ -0,0 +1,106 @@ +package report + +import "strings" + +// SchemaVersion is the version of the JSON document shape emitted by WriteJSON. +// It appears as the top-level "schemaVersion" field of every document. +const SchemaVersion = "1.0" + +// Document is the UI-oriented JSON output of a successful validation run. +type Document struct { + SchemaVersion string `json:"schemaVersion"` + Meta Meta `json:"meta"` + Summary Summary `json:"summary"` + Groups []Group `json:"groups"` +} + +// Meta echoes the run inputs (never the apiKey) so a UI can show what was checked. +type Meta struct { + GeneratedAt string `json:"generatedAt"` // RFC3339, UTC + OBAServerURL string `json:"obaServerURL"` + DataSources []MetaSource `json:"dataSources"` +} + +// MetaSource echoes one configured data source. ID joins to Group.ID. +type MetaSource struct { + ID string `json:"id"` + Index int `json:"index"` + StaticGtfsFeedURL string `json:"staticGtfsFeedURL,omitempty"` + VehiclePositionsURL string `json:"vehiclePositionsURL,omitempty"` + TripUpdatesURL string `json:"tripUpdatesURL,omitempty"` + ServiceAlertsURL string `json:"serviceAlertsURL,omitempty"` + AgencyMapping map[string]string `json:"agencyMapping,omitempty"` +} + +// Summary is the run-wide verdict and tallies. +type Summary struct { + Verdict string `json:"verdict"` // PASS | FAIL + ExitCode int `json:"exitCode"` + Total int `json:"total"` + Counts Counts `json:"counts"` +} + +// Counts tallies results by status. Keys are lowercase; values always present. +type Counts struct { + Pass int `json:"pass"` + Warn int `json:"warn"` + Fail int `json:"fail"` + Skip int `json:"skip"` +} + +// add increments the tally for an uppercase status string (PASS/WARN/FAIL/SKIP). +func (c *Counts) add(status string) { + switch status { + case "PASS": + c.Pass++ + case "WARN": + c.Warn++ + case "FAIL": + c.Fail++ + case "SKIP": + c.Skip++ + } +} + +// Group is one section of results: the server, or one data source. +type Group struct { + ID string `json:"id"` + Label string `json:"label"` + Counts Counts `json:"counts"` + Results []Item `json:"results"` +} + +// Item is one check result, with the check name pre-split into category/step. +type Item struct { + Check string `json:"check"` + Category string `json:"category"` + Step string `json:"step,omitempty"` + Status string `json:"status"` + Message string `json:"message"` + Details map[string]any `json:"details,omitempty"` +} + +// ErrorDocument is emitted to stdout when a --json run fails before producing a report. +type ErrorDocument struct { + SchemaVersion string `json:"schemaVersion"` + Error string `json:"error"` +} + +// splitCheck splits a check name on the first '/' into category and step. +// A name without '/' yields (name, ""). +func splitCheck(check string) (category, step string) { + if i := strings.IndexByte(check, '/'); i >= 0 { + return check[:i], check[i+1:] + } + return check, "" +} + +// redactString replaces the apiKey substring with "***" (matching the +// validator's redact convention) so a secret never reaches output. A no-op when +// apiKey is empty. +func redactString(s, apiKey string) string { + if apiKey == "" { + return s + } + return strings.ReplaceAll(s, apiKey, "***") +} diff --git a/report/document_test.go b/report/document_test.go new file mode 100644 index 0000000..e384081 --- /dev/null +++ b/report/document_test.go @@ -0,0 +1,36 @@ +package report + +import ( + "testing" +) + +func TestSplitCheck(t *testing.T) { + cases := []struct { + in string + category string + step string + }{ + {"basic-endpoints/current-time", "basic-endpoints", "current-time"}, + {"vehicle-positions-sampling/trip-for-vehicle", "vehicle-positions-sampling", "trip-for-vehicle"}, + {"agency-union", "agency-union", ""}, + {"a/b/c", "a", "b/c"}, + } + for _, c := range cases { + cat, step := splitCheck(c.in) + if cat != c.category || step != c.step { + t.Errorf("splitCheck(%q) = (%q,%q) want (%q,%q)", c.in, cat, step, c.category, c.step) + } + } +} + +func TestRedactString(t *testing.T) { + if got := redactString("https://x/?key=SEKRET", "SEKRET"); got != "https://x/?key=***" { + t.Errorf("redactString did not redact: %q", got) + } + if got := redactString("no-key-here", "SEKRET"); got != "no-key-here" { + t.Errorf("redactString altered non-matching string: %q", got) + } + if got := redactString("anything", ""); got != "anything" { + t.Errorf("empty apiKey should be a no-op: %q", got) + } +} From 5d1fed80255906d6302bf43c8f4c5f47e892832d Mon Sep 17 00:00:00 2001 From: Aaron Brethorst Date: Mon, 25 May 2026 13:48:18 -0700 Subject: [PATCH 04/10] feat(report): BuildDocument transform and UI-shaped WriteJSON/WriteErrorJSON Merges plan Tasks 2 and 3: the shared sampleReport test fixture changes shape, so the BuildDocument transform and the report_test.go rewrite must land together to keep the build green. --- cmd/oba-validator/main.go | 2 +- report/document.go | 103 +++++++++++++++++++++++++++++++++- report/document_test.go | 113 ++++++++++++++++++++++++++++++++++++++ report/report.go | 15 ++++- report/report_test.go | 47 +++++++++------- 5 files changed, 256 insertions(+), 24 deletions(-) diff --git a/cmd/oba-validator/main.go b/cmd/oba-validator/main.go index 39faf64..9b6609d 100644 --- a/cmd/oba-validator/main.go +++ b/cmd/oba-validator/main.go @@ -81,7 +81,7 @@ func run(args []string, stdout, stderr io.Writer) int { var werr error if o.jsonOut { - werr = report.WriteJSON(stdout, rep) + werr = report.WriteJSON(stdout, rep, cfg) } else { werr = report.WriteText(stdout, rep) } diff --git a/report/document.go b/report/document.go index f87b4dd..2cf5d30 100644 --- a/report/document.go +++ b/report/document.go @@ -1,6 +1,14 @@ package report -import "strings" +import ( + "fmt" + "sort" + "strings" + "time" + + "github.com/onebusaway/oba-validator/config" + "github.com/onebusaway/oba-validator/validator" +) // SchemaVersion is the version of the JSON document shape emitted by WriteJSON. // It appears as the top-level "schemaVersion" field of every document. @@ -104,3 +112,96 @@ func redactString(s, apiKey string) string { } return strings.ReplaceAll(s, apiKey, "***") } + +// BuildDocument transforms a validation report and its config into the +// UI-oriented Document. It is pure: pass time.Now().UTC() for now in production +// and a fixed time in tests. The apiKey is never echoed; URLs are redacted. +func BuildDocument(rep validator.Report, cfg config.Config, now time.Time) Document { + bySource := map[string][]validator.Result{} + for _, r := range rep.Results { + bySource[r.Source] = append(bySource[r.Source], r) + } + + groups := []Group{buildGroup("server", "Server", bySource[""])} + delete(bySource, "") + for i := range cfg.DataSources { + id := fmt.Sprintf("dataSource[%d]", i) + groups = append(groups, buildGroup(id, fmt.Sprintf("Data source %d", i), bySource[id])) + delete(bySource, id) + } + // Any result with an unrecognized source is emitted in a trailing group + // (sorted) so no data is ever dropped. + leftover := make([]string, 0, len(bySource)) + for k := range bySource { + leftover = append(leftover, k) + } + sort.Strings(leftover) + for _, k := range leftover { + groups = append(groups, buildGroup(k, k, bySource[k])) + } + + return Document{ + SchemaVersion: SchemaVersion, + Meta: buildMeta(cfg, now), + Summary: buildSummary(rep, groups), + Groups: groups, + } +} + +func buildGroup(id, label string, results []validator.Result) Group { + g := Group{ID: id, Label: label, Results: []Item{}} + for _, r := range results { + cat, step := splitCheck(r.Check) + status := r.Status.String() + g.Results = append(g.Results, Item{ + Check: r.Check, + Category: cat, + Step: step, + Status: status, + Message: r.Message, + Details: r.Details, + }) + g.Counts.add(status) + } + return g +} + +func buildMeta(cfg config.Config, now time.Time) Meta { + m := Meta{ + GeneratedAt: now.UTC().Format(time.RFC3339), + OBAServerURL: redactString(cfg.OBAServerURL, cfg.APIKey), + DataSources: make([]MetaSource, 0, len(cfg.DataSources)), + } + for i, ds := range cfg.DataSources { + m.DataSources = append(m.DataSources, MetaSource{ + ID: fmt.Sprintf("dataSource[%d]", i), + Index: i, + StaticGtfsFeedURL: redactString(ds.StaticGtfsFeedURL, cfg.APIKey), + VehiclePositionsURL: redactString(ds.VehiclePositionsURL, cfg.APIKey), + TripUpdatesURL: redactString(ds.TripUpdatesURL, cfg.APIKey), + ServiceAlertsURL: redactString(ds.ServiceAlertsURL, cfg.APIKey), + AgencyMapping: ds.AgencyMapping, + }) + } + return m +} + +func buildSummary(rep validator.Report, groups []Group) Summary { + var total Counts + for _, g := range groups { + total.Pass += g.Counts.Pass + total.Warn += g.Counts.Warn + total.Fail += g.Counts.Fail + total.Skip += g.Counts.Skip + } + verdict := "PASS" + if rep.Worst() == validator.Fail { + verdict = "FAIL" + } + return Summary{ + Verdict: verdict, + ExitCode: rep.ExitCode(), + Total: total.Pass + total.Warn + total.Fail + total.Skip, + Counts: total, + } +} diff --git a/report/document_test.go b/report/document_test.go index e384081..4b1b32b 100644 --- a/report/document_test.go +++ b/report/document_test.go @@ -1,7 +1,13 @@ package report import ( + "encoding/json" + "strings" "testing" + "time" + + "github.com/onebusaway/oba-validator/config" + "github.com/onebusaway/oba-validator/validator" ) func TestSplitCheck(t *testing.T) { @@ -34,3 +40,110 @@ func TestRedactString(t *testing.T) { t.Errorf("empty apiKey should be a no-op: %q", got) } } + +func fixedTime() time.Time { return time.Date(2026, 5, 25, 17, 4, 0, 0, time.UTC) } + +func sampleReport() validator.Report { + return validator.Report{Results: []validator.Result{ + {Check: "basic-endpoints/current-time", Status: validator.Pass, Message: "OK"}, + {Check: "agency-union", Status: validator.Pass, Message: "all agencies present"}, + {Check: "vehicle-positions-sampling/trip-for-vehicle", Source: "dataSource[0]", Status: validator.Fail, Message: "missing", Details: map[string]any{"vehicleId": "1_1234"}}, + {Check: "freshness", Source: "dataSource[0]", Status: validator.Warn, Message: "empty feed"}, + }} +} + +func sampleConfig() config.Config { + return config.Config{ + OBAServerURL: "https://oba.example.org", + APIKey: "secret-key", + DataSources: []config.DataSource{{ + StaticGtfsFeedURL: "https://feeds.example.org/gtfs.zip", + VehiclePositionsURL: "https://feeds.example.org/vp.pb", + AgencyMapping: map[string]string{"KCM": "1"}, + }}, + } +} + +func TestBuildDocument_GroupingAndOrder(t *testing.T) { + doc := BuildDocument(sampleReport(), sampleConfig(), fixedTime()) + if len(doc.Groups) != 2 { + t.Fatalf("groups=%d want 2", len(doc.Groups)) + } + if doc.Groups[0].ID != "server" || doc.Groups[0].Label != "Server" { + t.Errorf("group[0]=%+v want server/Server", doc.Groups[0]) + } + if doc.Groups[1].ID != "dataSource[0]" || doc.Groups[1].Label != "Data source 0" { + t.Errorf("group[1]=%+v want dataSource[0]/Data source 0", doc.Groups[1]) + } + if len(doc.Groups[0].Results) != 2 || len(doc.Groups[1].Results) != 2 { + t.Errorf("group sizes = %d,%d want 2,2", len(doc.Groups[0].Results), len(doc.Groups[1].Results)) + } +} + +func TestBuildDocument_CategoryStep(t *testing.T) { + doc := BuildDocument(sampleReport(), sampleConfig(), fixedTime()) + got := doc.Groups[0].Results + if got[0].Category != "basic-endpoints" || got[0].Step != "current-time" { + t.Errorf("result[0] cat/step = %q/%q", got[0].Category, got[0].Step) + } + if got[1].Category != "agency-union" || got[1].Step != "" { + t.Errorf("result[1] cat/step = %q/%q want agency-union/empty", got[1].Category, got[1].Step) + } +} + +func TestBuildDocument_CountsVerdictExit(t *testing.T) { + doc := BuildDocument(sampleReport(), sampleConfig(), fixedTime()) + if doc.Groups[0].Counts != (Counts{Pass: 2}) { + t.Errorf("server counts=%+v", doc.Groups[0].Counts) + } + if doc.Groups[1].Counts != (Counts{Fail: 1, Warn: 1}) { + t.Errorf("ds counts=%+v", doc.Groups[1].Counts) + } + if doc.Summary.Counts != (Counts{Pass: 2, Warn: 1, Fail: 1}) { + t.Errorf("summary counts=%+v", doc.Summary.Counts) + } + if doc.Summary.Total != 4 { + t.Errorf("total=%d want 4", doc.Summary.Total) + } + if doc.Summary.Verdict != "FAIL" || doc.Summary.ExitCode != 1 { + t.Errorf("verdict/exit = %q/%d want FAIL/1", doc.Summary.Verdict, doc.Summary.ExitCode) + } + if doc.SchemaVersion != SchemaVersion { + t.Errorf("schemaVersion=%q", doc.SchemaVersion) + } +} + +func TestBuildDocument_MetaEcho(t *testing.T) { + doc := BuildDocument(sampleReport(), sampleConfig(), fixedTime()) + if doc.Meta.GeneratedAt != "2026-05-25T17:04:00Z" { + t.Errorf("generatedAt=%q", doc.Meta.GeneratedAt) + } + if doc.Meta.OBAServerURL != "https://oba.example.org" { + t.Errorf("obaServerURL=%q", doc.Meta.OBAServerURL) + } + if len(doc.Meta.DataSources) != 1 { + t.Fatalf("meta dataSources=%d want 1", len(doc.Meta.DataSources)) + } + ms := doc.Meta.DataSources[0] + if ms.ID != "dataSource[0]" || ms.Index != 0 || ms.AgencyMapping["KCM"] != "1" { + t.Errorf("metaSource=%+v", ms) + } + // tripUpdatesURL/serviceAlertsURL were not configured -> omitted from JSON. + b, _ := json.Marshal(doc) + if strings.Contains(string(b), "tripUpdatesURL") || strings.Contains(string(b), "serviceAlertsURL") { + t.Errorf("unconfigured URLs should be omitted:\n%s", b) + } +} + +func TestBuildDocument_RedactsAPIKey(t *testing.T) { + cfg := sampleConfig() + cfg.APIKey = "SEKRET" + cfg.OBAServerURL = "https://oba.example.org/?key=SEKRET" + b, _ := json.Marshal(BuildDocument(sampleReport(), cfg, fixedTime())) + if strings.Contains(string(b), "SEKRET") { + t.Errorf("apiKey leaked into output:\n%s", b) + } + if !strings.Contains(string(b), "***") { + t.Errorf("expected redaction marker in output:\n%s", b) + } +} diff --git a/report/report.go b/report/report.go index 97dc3e5..c8fa768 100644 --- a/report/report.go +++ b/report/report.go @@ -5,15 +5,24 @@ import ( "encoding/json" "fmt" "io" + "time" + "github.com/onebusaway/oba-validator/config" "github.com/onebusaway/oba-validator/validator" ) -// WriteJSON writes the report as indented JSON. -func WriteJSON(w io.Writer, rep validator.Report) error { +// WriteJSON writes the report as an indented, UI-oriented JSON Document. +func WriteJSON(w io.Writer, rep validator.Report, cfg config.Config) error { enc := json.NewEncoder(w) enc.SetIndent("", " ") - return enc.Encode(rep) + return enc.Encode(BuildDocument(rep, cfg, time.Now().UTC())) +} + +// WriteErrorJSON writes an indented ErrorDocument to w, redacting apiKey from msg. +func WriteErrorJSON(w io.Writer, msg, apiKey string) error { + enc := json.NewEncoder(w) + enc.SetIndent("", " ") + return enc.Encode(ErrorDocument{SchemaVersion: SchemaVersion, Error: redactString(msg, apiKey)}) } // WriteText writes a human-readable, grouped report with a summary line. diff --git a/report/report_test.go b/report/report_test.go index 6eec073..b32bcaa 100644 --- a/report/report_test.go +++ b/report/report_test.go @@ -5,33 +5,42 @@ import ( "encoding/json" "strings" "testing" - - "github.com/onebusaway/oba-validator/validator" ) -func sampleReport() validator.Report { - return validator.Report{Results: []validator.Result{ - {Check: "basic-endpoints/current-time", Status: validator.Pass, Message: "OK"}, - {Check: "vehicle-positions-sampling", Source: "dataSource[0]", Status: validator.Fail, Message: "missing"}, - }} -} - func TestWriteJSON(t *testing.T) { var buf bytes.Buffer - if err := WriteJSON(&buf, sampleReport()); err != nil { + if err := WriteJSON(&buf, sampleReport(), sampleConfig()); err != nil { t.Fatal(err) } - var back struct { - Results []struct { - Check string `json:"check"` - Status string `json:"status"` - } `json:"results"` + var doc Document + if err := json.Unmarshal(buf.Bytes(), &doc); err != nil { + t.Fatalf("output not a Document: %v\n%s", err, buf.String()) + } + if doc.SchemaVersion != SchemaVersion { + t.Errorf("schemaVersion=%q", doc.SchemaVersion) + } + if len(doc.Groups) != 2 || doc.Summary.Verdict != "FAIL" { + t.Errorf("unexpected document: %+v", doc.Summary) + } + if !strings.Contains(buf.String(), "\n ") { + t.Error("expected indented JSON") } - if err := json.Unmarshal(buf.Bytes(), &back); err != nil { +} + +func TestWriteErrorJSON(t *testing.T) { + var buf bytes.Buffer + if err := WriteErrorJSON(&buf, "boom with SEKRET", "SEKRET"); err != nil { t.Fatal(err) } - if back.Results[1].Status != "FAIL" { - t.Errorf("status=%q want FAIL", back.Results[1].Status) + var ed ErrorDocument + if err := json.Unmarshal(buf.Bytes(), &ed); err != nil { + t.Fatalf("output not an ErrorDocument: %v\n%s", err, buf.String()) + } + if ed.SchemaVersion != SchemaVersion { + t.Errorf("schemaVersion=%q", ed.SchemaVersion) + } + if strings.Contains(ed.Error, "SEKRET") || !strings.Contains(ed.Error, "***") { + t.Errorf("error not redacted: %q", ed.Error) } } @@ -54,7 +63,7 @@ func TestWriteTextSummaryLine(t *testing.T) { if err := WriteText(&buf, sampleReport()); err != nil { t.Fatal(err) } - if !strings.Contains(buf.String(), "FAIL (2 checks, 1 failed, 0 warnings)") { + if !strings.Contains(buf.String(), "FAIL (4 checks, 1 failed, 1 warnings)") { t.Errorf("summary line wrong:\n%s", buf.String()) } } From b4b86aa6395cc33c14545590420c250448492b9f Mon Sep 17 00:00:00 2001 From: Aaron Brethorst Date: Mon, 25 May 2026 13:48:43 -0700 Subject: [PATCH 05/10] feat(cli): emit JSON error document on failure under --json --- cmd/oba-validator/main.go | 16 ++++++++-- cmd/oba-validator/main_test.go | 55 ++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+), 2 deletions(-) diff --git a/cmd/oba-validator/main.go b/cmd/oba-validator/main.go index 9b6609d..40c1cb0 100644 --- a/cmd/oba-validator/main.go +++ b/cmd/oba-validator/main.go @@ -68,14 +68,26 @@ func run(args []string, stdout, stderr io.Writer) int { cfg, err := config.Load(fs.Arg(0)) if err != nil { - fmt.Fprintln(stderr, "config error:", err) + if o.jsonOut { + if werr := report.WriteErrorJSON(stdout, err.Error(), os.Getenv("ONEBUSAWAY_API_KEY")); werr != nil { + fmt.Fprintln(stderr, "output error:", werr) + } + } else { + fmt.Fprintln(stderr, "config error:", err) + } return 2 } applyOverrides(&cfg, o) rep, err := validator.Run(context.Background(), cfg) if err != nil { - fmt.Fprintln(stderr, "run error:", err) + if o.jsonOut { + if werr := report.WriteErrorJSON(stdout, err.Error(), cfg.APIKey); werr != nil { + fmt.Fprintln(stderr, "output error:", werr) + } + } else { + fmt.Fprintln(stderr, "run error:", err) + } return 2 } diff --git a/cmd/oba-validator/main_test.go b/cmd/oba-validator/main_test.go index b444666..a92e3c0 100644 --- a/cmd/oba-validator/main_test.go +++ b/cmd/oba-validator/main_test.go @@ -2,6 +2,10 @@ package main import ( "bytes" + "encoding/json" + "net/http" + "net/http/httptest" + "strings" "testing" "github.com/onebusaway/oba-validator/config" @@ -36,3 +40,54 @@ func TestUsageWhenTooManyArgs(t *testing.T) { t.Error("expected usage on stderr") } } + +func TestRunJSONConfigErrorEmitsErrorJSON(t *testing.T) { + var stdout, stderr bytes.Buffer + code := run([]string{"oba-validator", "--json", `{"dataSources":[]}`}, &stdout, &stderr) + if code != 2 { + t.Fatalf("exit=%d want 2", code) + } + var ed struct { + SchemaVersion string `json:"schemaVersion"` + Error string `json:"error"` + } + if err := json.Unmarshal(stdout.Bytes(), &ed); err != nil { + t.Fatalf("stdout not JSON: %v\n%s", err, stdout.String()) + } + if ed.Error == "" || ed.SchemaVersion == "" { + t.Errorf("missing fields in error doc: %s", stdout.String()) + } +} + +func TestRunJSONOutputShape(t *testing.T) { + obaSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + switch { + case strings.Contains(r.URL.Path, "current-time"): + w.Write([]byte(`{"data":{"entry":{"time":1716000000000}}}`)) + case strings.Contains(r.URL.Path, "agencies-with-coverage"): + w.Write([]byte(`{"data":{"list":[],"references":{"agencies":[]}}}`)) + default: + w.Write([]byte(`{"data":{"list":[],"entry":{"arrivalsAndDepartures":[]}}}`)) + } + })) + defer obaSrv.Close() + feedSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Write([]byte{}) // empty payload -> prep error recorded, run still completes + })) + defer feedSrv.Close() + + cfg := `{"obaServerURL":"` + obaSrv.URL + `","apiKey":"test","dataSources":[{"staticGtfsFeedURL":"` + feedSrv.URL + `/gtfs.zip"}]}` + var stdout, stderr bytes.Buffer + run([]string{"oba-validator", "--json", "--no-cache", cfg}, &stdout, &stderr) + + var doc map[string]json.RawMessage + if err := json.Unmarshal(stdout.Bytes(), &doc); err != nil { + t.Fatalf("stdout not JSON: %v\n%s", err, stdout.String()) + } + for _, k := range []string{"schemaVersion", "meta", "summary", "groups"} { + if _, ok := doc[k]; !ok { + t.Errorf("missing key %q in output:\n%s", k, stdout.String()) + } + } +} From 16a2083ac7a3484e2c789ebd8e451a0b8c14c776 Mon Sep 17 00:00:00 2001 From: Aaron Brethorst Date: Mon, 25 May 2026 13:49:18 -0700 Subject: [PATCH 06/10] feat: publish JSON Schema and conformance test for --json output --- go.mod | 3 +- go.sum | 8 +- report/schema_test.go | 70 +++++++++++++++ schema/oba-validator-report.schema.json | 110 ++++++++++++++++++++++++ 4 files changed, 188 insertions(+), 3 deletions(-) create mode 100644 report/schema_test.go create mode 100644 schema/oba-validator-report.schema.json diff --git a/go.mod b/go.mod index 4a84ed1..bcc3a7b 100644 --- a/go.mod +++ b/go.mod @@ -5,6 +5,7 @@ go 1.25.9 require ( github.com/OneBusAway/go-gtfs v1.1.1 github.com/OneBusAway/go-sdk v1.11.0 + github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 ) require ( @@ -12,6 +13,6 @@ require ( github.com/tidwall/match v1.1.1 // indirect github.com/tidwall/pretty v1.2.1 // indirect github.com/tidwall/sjson v1.2.5 // indirect - golang.org/x/text v0.9.0 // indirect + golang.org/x/text v0.14.0 // indirect google.golang.org/protobuf v1.27.1 // indirect ) diff --git a/go.sum b/go.sum index 8029e7d..9ddc01b 100644 --- a/go.sum +++ b/go.sum @@ -2,10 +2,14 @@ github.com/OneBusAway/go-gtfs v1.1.1 h1:JWl0ndXHBED6PAh8v3w0UgSDYWBg2OmHvAJb5RXX github.com/OneBusAway/go-gtfs v1.1.1/go.mod h1:MJqNyFOJs+iE1R6uerTyfBY6g3/sxvTvVdRhDeN1bu8= github.com/OneBusAway/go-sdk v1.11.0 h1:gC7T7x0DgGSV1bd11eSu1cnQLriQLgEo+0Cx3dqhcuo= github.com/OneBusAway/go-sdk v1.11.0/go.mod h1:7Rj5b+lGJROO+UqrkHPEjwJcXddbhwL0CQSJrLaAWSA= +github.com/dlclark/regexp2 v1.11.0 h1:G/nrcoOa7ZXlpoa/91N3X7mM3r8eIlMBBJZvsz/mxKI= +github.com/dlclark/regexp2 v1.11.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8= github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk= github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38= github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= +github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 h1:KRzFb2m7YtdldCEkzs6KqmJw4nqEVZGK7IN2kJkjTuQ= +github.com/santhosh-tekuri/jsonschema/v6 v6.0.2/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU= github.com/tidwall/gjson v1.14.2/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk= github.com/tidwall/gjson v1.14.4 h1:uo0p8EbA09J7RQaflQ1aBRffTR7xedD2bcIVSYxLnkM= github.com/tidwall/gjson v1.14.4/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk= @@ -16,8 +20,8 @@ github.com/tidwall/pretty v1.2.1 h1:qjsOFOWWQl+N3RsoF5/ssm1pHmJJwhjlSbZ51I6wMl4= github.com/tidwall/pretty v1.2.1/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU= github.com/tidwall/sjson v1.2.5 h1:kLy8mja+1c9jlljvWTlSazM7cKDRfJuR/bOJhcY5NcY= github.com/tidwall/sjson v1.2.5/go.mod h1:Fvgq9kS/6ociJEDnK0Fk1cpYF4FIW6ZF7LAe+6jwd28= -golang.org/x/text v0.9.0 h1:2sjJmO8cDvYveuX97RDLsxlyUxLl+GHoLxBiRdHllBE= -golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8= +golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ= +golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw= google.golang.org/protobuf v1.27.1 h1:SnqbnDw1V7RiZcXPx5MEeqPv2s79L9i7BJUlG/+RurQ= diff --git a/report/schema_test.go b/report/schema_test.go new file mode 100644 index 0000000..199b094 --- /dev/null +++ b/report/schema_test.go @@ -0,0 +1,70 @@ +package report + +import ( + "encoding/json" + "os" + "path/filepath" + "testing" + + "github.com/santhosh-tekuri/jsonschema/v6" +) + +func compileSchema(t *testing.T) *jsonschema.Schema { + t.Helper() + path := filepath.Join("..", "schema", "oba-validator-report.schema.json") + data, err := os.ReadFile(path) + if err != nil { + t.Fatalf("read schema: %v", err) + } + var doc any + if err := json.Unmarshal(data, &doc); err != nil { + t.Fatalf("schema is not valid JSON: %v", err) + } + c := jsonschema.NewCompiler() + if err := c.AddResource("report.json", doc); err != nil { + t.Fatalf("add resource: %v", err) + } + sch, err := c.Compile("report.json") + if err != nil { + t.Fatalf("compile schema: %v", err) + } + return sch +} + +func validateAgainst(t *testing.T, sch *jsonschema.Schema, v any) error { + t.Helper() + b, err := json.Marshal(v) + if err != nil { + t.Fatal(err) + } + var inst any + if err := json.Unmarshal(b, &inst); err != nil { + t.Fatal(err) + } + return sch.Validate(inst) +} + +func TestSchema_SuccessDocumentConforms(t *testing.T) { + sch := compileSchema(t) + if err := validateAgainst(t, sch, BuildDocument(sampleReport(), sampleConfig(), fixedTime())); err != nil { + t.Errorf("success document failed schema:\n%v", err) + } +} + +func TestSchema_ErrorDocumentConforms(t *testing.T) { + sch := compileSchema(t) + ed := ErrorDocument{SchemaVersion: SchemaVersion, Error: "boom"} + if err := validateAgainst(t, sch, ed); err != nil { + t.Errorf("error document failed schema:\n%v", err) + } +} + +func TestSchema_RejectsMalformed(t *testing.T) { + sch := compileSchema(t) + // Has schemaVersion but is neither a valid report (missing meta/summary/groups) + // nor a valid error (no "error"); oneOf must match zero variants. + bad := map[string]any{"schemaVersion": "1.0", "summary": map[string]any{}} + if err := validateAgainst(t, sch, bad); err == nil { + t.Error("expected malformed document to fail schema validation") + } +} diff --git a/schema/oba-validator-report.schema.json b/schema/oba-validator-report.schema.json new file mode 100644 index 0000000..0a45fee --- /dev/null +++ b/schema/oba-validator-report.schema.json @@ -0,0 +1,110 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://github.com/onebusaway/oba-validator/schema/oba-validator-report.schema.json", + "title": "OBA Validator Output", + "description": "Output of `oba-validator --json`: either a validation report document or an error document. Both carry a top-level schemaVersion.", + "oneOf": [ + { "$ref": "#/$defs/reportDocument" }, + { "$ref": "#/$defs/errorDocument" } + ], + "$defs": { + "status": { + "type": "string", + "enum": ["PASS", "WARN", "FAIL", "SKIP"] + }, + "counts": { + "type": "object", + "description": "Result tallies by status.", + "additionalProperties": false, + "required": ["pass", "warn", "fail", "skip"], + "properties": { + "pass": { "type": "integer", "minimum": 0 }, + "warn": { "type": "integer", "minimum": 0 }, + "fail": { "type": "integer", "minimum": 0 }, + "skip": { "type": "integer", "minimum": 0 } + } + }, + "item": { + "type": "object", + "description": "One check result. category/step are the check name split on the first '/'.", + "additionalProperties": false, + "required": ["check", "category", "status", "message"], + "properties": { + "check": { "type": "string" }, + "category": { "type": "string" }, + "step": { "type": "string" }, + "status": { "$ref": "#/$defs/status" }, + "message": { "type": "string" }, + "details": { "type": "object", "additionalProperties": true } + } + }, + "group": { + "type": "object", + "description": "A section of results: the server, or one data source.", + "additionalProperties": false, + "required": ["id", "label", "counts", "results"], + "properties": { + "id": { "type": "string", "description": "'server' or 'dataSource[N]'; joins to meta.dataSources[].id." }, + "label": { "type": "string" }, + "counts": { "$ref": "#/$defs/counts" }, + "results": { "type": "array", "items": { "$ref": "#/$defs/item" } } + } + }, + "metaSource": { + "type": "object", + "description": "Echo of one configured data source. The apiKey is never present.", + "additionalProperties": false, + "required": ["id", "index"], + "properties": { + "id": { "type": "string" }, + "index": { "type": "integer", "minimum": 0 }, + "staticGtfsFeedURL": { "type": "string" }, + "vehiclePositionsURL": { "type": "string" }, + "tripUpdatesURL": { "type": "string" }, + "serviceAlertsURL": { "type": "string" }, + "agencyMapping": { "type": "object", "additionalProperties": { "type": "string" } } + } + }, + "meta": { + "type": "object", + "additionalProperties": false, + "required": ["generatedAt", "obaServerURL", "dataSources"], + "properties": { + "generatedAt": { "type": "string", "format": "date-time" }, + "obaServerURL": { "type": "string" }, + "dataSources": { "type": "array", "items": { "$ref": "#/$defs/metaSource" } } + } + }, + "summary": { + "type": "object", + "additionalProperties": false, + "required": ["verdict", "exitCode", "total", "counts"], + "properties": { + "verdict": { "type": "string", "enum": ["PASS", "FAIL"] }, + "exitCode": { "type": "integer", "enum": [0, 1] }, + "total": { "type": "integer", "minimum": 0 }, + "counts": { "$ref": "#/$defs/counts" } + } + }, + "reportDocument": { + "type": "object", + "additionalProperties": false, + "required": ["schemaVersion", "meta", "summary", "groups"], + "properties": { + "schemaVersion": { "type": "string" }, + "meta": { "$ref": "#/$defs/meta" }, + "summary": { "$ref": "#/$defs/summary" }, + "groups": { "type": "array", "items": { "$ref": "#/$defs/group" } } + } + }, + "errorDocument": { + "type": "object", + "additionalProperties": false, + "required": ["schemaVersion", "error"], + "properties": { + "schemaVersion": { "type": "string" }, + "error": { "type": "string" } + } + } + } +} From cf942a4994397dc052364498ef885b20136ad758 Mon Sep 17 00:00:00 2001 From: Aaron Brethorst Date: Mon, 25 May 2026 13:49:43 -0700 Subject: [PATCH 07/10] docs: document --json UI output and schema --- CLAUDE.md | 2 +- README.md | 15 ++++++++++++++- .../specs/2026-05-25-json-ui-output-design.md | 2 +- 3 files changed, 16 insertions(+), 3 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index e16b814..4f74cf5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -32,7 +32,7 @@ The flow is **config → prepare (fetch) → checks → report**: 1. **`config`** — `config.Load()` accepts a file path *or* a raw JSON string (auto-detected by a leading `{`). Applies defaults, validates required fields, and reads `apiKey` from `ONEBUSAWAY_API_KEY` if absent. 2. **`feeds`** — fetching + parsing. `Fetcher` downloads feeds; static GTFS goes through an on-disk **conditional-GET `Cache`** (ETag/Last-Modified, atomic body-then-meta writes), realtime feeds are always fetched fresh. `ParsedStatic` wraps go-gtfs's `Static` with the lookup indexes checks need (agency IDs/names, raw trip→agency, raw route→agency). 3. **`validator`** — the engine. `validator.Run()` calls `prepare()`, then runs every check. -4. **`report`** — renders a `Report` as grouped text (`WriteText`) or indented JSON (`WriteJSON`). +4. **`report`** — renders a `Report` as grouped text (`WriteText`) or, via `WriteJSON`, a UI-oriented JSON `Document` (meta + summary + grouped results; schema at `schema/oba-validator-report.schema.json`). `WriteErrorJSON` emits the error variant. The `Document` view model is built by the pure `BuildDocument(report, config, now)` so output is deterministic in tests. `prepare()` (`validator/validator.go`) builds the shared `ValidationContext`: it constructs the OBA SDK client, fetches `AgenciesWithCoverage` once, and **fans out concurrently** (bounded by `MaxConcurrency`, default 4) to download/parse each data source's feeds into a `SourceContext`. A per-feed fetch/parse failure is recorded in `SourceContext.PrepErrors[feedName]` rather than aborting the run — checks inspect that map and decide severity themselves. diff --git a/README.md b/README.md index f0cc94f..47f642d 100644 --- a/README.md +++ b/README.md @@ -41,10 +41,23 @@ to the `agencyId` the OBA server exposes; unmapped agencies default to identity. ```go cfg, _ := config.Load("config.json") // or a raw JSON string rep, _ := validator.Run(ctx, cfg) -report.WriteText(os.Stdout, rep) // or report.WriteJSON(os.Stdout, rep) +report.WriteText(os.Stdout, rep) // or report.WriteJSON(os.Stdout, rep, cfg) os.Exit(rep.ExitCode()) ``` +## JSON output + +`--json` emits a single structured document to stdout, designed for building a UI +visualization. It contains `meta` (run inputs — never the apiKey), `summary` +(verdict + status counts), and `groups` (a `server` group plus one per data +source, each with its results). On failure before a report is produced, a +`{ "schemaVersion", "error" }` object is emitted to stdout and the process exits 2. + +The full contract is published as a JSON Schema (draft 2020-12) at +[`schema/oba-validator-report.schema.json`](schema/oba-validator-report.schema.json). +This is the recommended format for the Render one-off-job workflow: the job +prints the document to stdout and the caller reads it from the job output. + ## Development make build # compile to bin/oba-validator diff --git a/docs/superpowers/specs/2026-05-25-json-ui-output-design.md b/docs/superpowers/specs/2026-05-25-json-ui-output-design.md index f09b88d..79681b1 100644 --- a/docs/superpowers/specs/2026-05-25-json-ui-output-design.md +++ b/docs/superpowers/specs/2026-05-25-json-ui-output-design.md @@ -1,7 +1,7 @@ # JSON-for-UI Output Mode — Design **Date:** 2026-05-25 -**Status:** Approved (design + both sign-off items); pending spec review +**Status:** Implemented ## Purpose From 375bb246669e36f641b4cee60d762ce821b38f1e Mon Sep 17 00:00:00 2001 From: Aaron Brethorst Date: Mon, 25 May 2026 13:55:59 -0700 Subject: [PATCH 08/10] fix(cli): redact inline apiKey from config-load error output A raw-JSON config argument that does not start with '{' (e.g. a JSON array or a BOM-prefixed object) is misread as a file path; config.Load's os.ReadFile error then echoes the raw input verbatim, including an inline apiKey. The --json path emitted this to stdout (text mode to stderr). Redact using a key sniffed from the argument itself, since the parsed cfg is empty when Load fails this way. --- cmd/oba-validator/main.go | 28 ++++++++++++++++++++++++++-- cmd/oba-validator/main_test.go | 30 ++++++++++++++++++++++++++++++ 2 files changed, 56 insertions(+), 2 deletions(-) diff --git a/cmd/oba-validator/main.go b/cmd/oba-validator/main.go index 40c1cb0..a051d20 100644 --- a/cmd/oba-validator/main.go +++ b/cmd/oba-validator/main.go @@ -6,12 +6,31 @@ import ( "fmt" "io" "os" + "regexp" + "strings" "github.com/onebusaway/oba-validator/config" "github.com/onebusaway/oba-validator/report" "github.com/onebusaway/oba-validator/validator" ) +// apiKeyInJSON matches an "apiKey" string field in a (possibly malformed) JSON +// argument so its value can be scrubbed from error output. +var apiKeyInJSON = regexp.MustCompile(`"apiKey"\s*:\s*"((?:\\.|[^"\\])*)"`) + +// redactionKey returns the apiKey to scrub from a config-load error. config.Load +// can fail before it parses the key — notably when a raw-JSON argument that does +// not start with '{' is misread as a file path, whose os.ReadFile error echoes +// the raw input (and thus an inline apiKey) verbatim. The parsed cfg is empty in +// that case, so prefer a key sniffed straight from the argument, falling back to +// the environment. +func redactionKey(arg string) string { + if m := apiKeyInJSON.FindStringSubmatch(arg); m != nil && m[1] != "" { + return m[1] + } + return os.Getenv("ONEBUSAWAY_API_KEY") +} + type overrides struct { jsonOut bool sampleSize int @@ -68,12 +87,17 @@ func run(args []string, stdout, stderr io.Writer) int { cfg, err := config.Load(fs.Arg(0)) if err != nil { + key := redactionKey(fs.Arg(0)) if o.jsonOut { - if werr := report.WriteErrorJSON(stdout, err.Error(), os.Getenv("ONEBUSAWAY_API_KEY")); werr != nil { + if werr := report.WriteErrorJSON(stdout, err.Error(), key); werr != nil { fmt.Fprintln(stderr, "output error:", werr) } } else { - fmt.Fprintln(stderr, "config error:", err) + msg := err.Error() + if key != "" { + msg = strings.ReplaceAll(msg, key, "***") + } + fmt.Fprintln(stderr, "config error:", msg) } return 2 } diff --git a/cmd/oba-validator/main_test.go b/cmd/oba-validator/main_test.go index a92e3c0..e15283f 100644 --- a/cmd/oba-validator/main_test.go +++ b/cmd/oba-validator/main_test.go @@ -91,3 +91,33 @@ func TestRunJSONOutputShape(t *testing.T) { } } } + +func TestRunJSONConfigErrorRedactsInlineAPIKey(t *testing.T) { + t.Setenv("ONEBUSAWAY_API_KEY", "") + var stdout, stderr bytes.Buffer + code := run([]string{"oba-validator", "--json", `[{"obaServerURL":"https://x","apiKey":"SUPER-SECRET-KEY"}]`}, &stdout, &stderr) + if code != 2 { + t.Fatalf("exit=%d want 2", code) + } + if strings.Contains(stdout.String(), "SUPER-SECRET-KEY") { + t.Errorf("apiKey leaked to stdout:\n%s", stdout.String()) + } + var ed struct { + Error string `json:"error"` + } + if err := json.Unmarshal(stdout.Bytes(), &ed); err != nil { + t.Fatalf("stdout not JSON: %v\n%s", err, stdout.String()) + } + if ed.Error == "" { + t.Errorf("expected an error message: %s", stdout.String()) + } +} + +func TestRunTextConfigErrorRedactsInlineAPIKey(t *testing.T) { + t.Setenv("ONEBUSAWAY_API_KEY", "") + var stdout, stderr bytes.Buffer + run([]string{"oba-validator", `[{"obaServerURL":"https://x","apiKey":"SUPER-SECRET-KEY"}]`}, &stdout, &stderr) + if strings.Contains(stderr.String(), "SUPER-SECRET-KEY") { + t.Errorf("apiKey leaked to stderr:\n%s", stderr.String()) + } +} From 6a7108389c3c9685faa7a4188fe91b4f205abbdd Mon Sep 17 00:00:00 2001 From: Aaron Brethorst Date: Mon, 25 May 2026 14:04:13 -0700 Subject: [PATCH 09/10] harden JSON output after multi-agent review - Counts.add takes typed validator.Status (compile-time enum coupling) and panics on an out-of-range value instead of silently dropping it; Summary.Total is now len(rep.Results) so it can't diverge from the source. - WriteJSON/WriteErrorJSON marshal fully then write once, so a mid-stream write failure can't leave partial JSON on the consumer's stream. - Redact result messages at the report layer (defense in depth; checks already redact upstream); redact the run-error text path for consistency. - Cover the dark branches and contracts: SKIP counting, leftover/unknown-source grouping, details passthrough/omit, zero-data-sources ([] not null), determinism, message redaction, and the unknown-status panic. - Comment/spec accuracy fixes. --- cmd/oba-validator/main.go | 15 ++-- .../specs/2026-05-25-json-ui-output-design.md | 8 +- report/document.go | 45 ++++++---- report/document_test.go | 87 +++++++++++++++++++ report/report.go | 23 +++-- 5 files changed, 143 insertions(+), 35 deletions(-) diff --git a/cmd/oba-validator/main.go b/cmd/oba-validator/main.go index a051d20..b8e2d31 100644 --- a/cmd/oba-validator/main.go +++ b/cmd/oba-validator/main.go @@ -19,10 +19,11 @@ import ( var apiKeyInJSON = regexp.MustCompile(`"apiKey"\s*:\s*"((?:\\.|[^"\\])*)"`) // redactionKey returns the apiKey to scrub from a config-load error. config.Load -// can fail before it parses the key — notably when a raw-JSON argument that does -// not start with '{' is misread as a file path, whose os.ReadFile error echoes -// the raw input (and thus an inline apiKey) verbatim. The parsed cfg is empty in -// that case, so prefer a key sniffed straight from the argument, falling back to +// can fail before it parses the key and echo the raw argument (and thus an inline +// apiKey) into its error — either when a raw-JSON argument that does not start +// with '{' is misread as a file path (the os.ReadFile error wraps the input), or +// when a malformed object fails to parse. config.Load returns an empty Config in +// both cases, so prefer a key sniffed straight from the argument, falling back to // the environment. func redactionKey(arg string) string { if m := apiKeyInJSON.FindStringSubmatch(arg); m != nil && m[1] != "" { @@ -110,7 +111,11 @@ func run(args []string, stdout, stderr io.Writer) int { fmt.Fprintln(stderr, "output error:", werr) } } else { - fmt.Fprintln(stderr, "run error:", err) + msg := err.Error() + if cfg.APIKey != "" { + msg = strings.ReplaceAll(msg, cfg.APIKey, "***") + } + fmt.Fprintln(stderr, "run error:", msg) } return 2 } diff --git a/docs/superpowers/specs/2026-05-25-json-ui-output-design.md b/docs/superpowers/specs/2026-05-25-json-ui-output-design.md index 79681b1..4d13010 100644 --- a/docs/superpowers/specs/2026-05-25-json-ui-output-design.md +++ b/docs/superpowers/specs/2026-05-25-json-ui-output-design.md @@ -137,10 +137,10 @@ validator.Run ──err──▶ (--json? WriteErrorJSON : text) ; exit 2 ### Security `apiKey` is never placed in the document. As defense in depth, every echoed URL -in `meta` is passed through a redactor that replaces the apiKey substring with -`REDACTED` if it ever appears (mirrors the existing `redact(err, key)` rule in -the validator). `Details` values pass through unchanged — checks are already -responsible for redacting them at the source. +in `meta` **and every result message** is passed through a redactor that replaces +the apiKey substring with `***` if it ever appears (mirrors the existing +`redact(err, key)` rule in the validator). `Details` values pass through +unchanged — checks are already responsible for redacting them at the source. ## Output shape (error) diff --git a/report/document.go b/report/document.go index 2cf5d30..86dd265 100644 --- a/report/document.go +++ b/report/document.go @@ -48,7 +48,8 @@ type Summary struct { Counts Counts `json:"counts"` } -// Counts tallies results by status. Keys are lowercase; values always present. +// Counts tallies results by status. All four fields are always serialized (no +// omitempty), so a UI can rely on every count being present, including zeros. type Counts struct { Pass int `json:"pass"` Warn int `json:"warn"` @@ -56,17 +57,21 @@ type Counts struct { Skip int `json:"skip"` } -// add increments the tally for an uppercase status string (PASS/WARN/FAIL/SKIP). -func (c *Counts) add(status string) { +// add increments the tally for a status. It takes the typed validator.Status so +// the compiler keeps this switch in sync with the enum; an out-of-range value is +// a programming error and panics rather than being silently dropped. +func (c *Counts) add(status validator.Status) { switch status { - case "PASS": + case validator.Pass: c.Pass++ - case "WARN": + case validator.Warn: c.Warn++ - case "FAIL": + case validator.Fail: c.Fail++ - case "SKIP": + case validator.Skip: c.Skip++ + default: + panic(fmt.Sprintf("report: unexpected status %v", status)) } } @@ -104,8 +109,8 @@ func splitCheck(check string) (category, step string) { } // redactString replaces the apiKey substring with "***" (matching the -// validator's redact convention) so a secret never reaches output. A no-op when -// apiKey is empty. +// validator's redact convention in validator/util.go — keep the token in sync) +// so a secret never reaches output. A no-op when apiKey is empty. func redactString(s, apiKey string) string { if apiKey == "" { return s @@ -115,18 +120,21 @@ func redactString(s, apiKey string) string { // BuildDocument transforms a validation report and its config into the // UI-oriented Document. It is pure: pass time.Now().UTC() for now in production -// and a fixed time in tests. The apiKey is never echoed; URLs are redacted. +// and a fixed time in tests. The apiKey is never copied into the Document, and +// any occurrence of cfg.APIKey in echoed URLs and result messages is replaced +// with "***" (a no-op when the key is empty). Result Details pass through +// unchanged — checks are responsible for redacting them at the source. func BuildDocument(rep validator.Report, cfg config.Config, now time.Time) Document { bySource := map[string][]validator.Result{} for _, r := range rep.Results { bySource[r.Source] = append(bySource[r.Source], r) } - groups := []Group{buildGroup("server", "Server", bySource[""])} + groups := []Group{buildGroup("server", "Server", bySource[""], cfg.APIKey)} delete(bySource, "") for i := range cfg.DataSources { id := fmt.Sprintf("dataSource[%d]", i) - groups = append(groups, buildGroup(id, fmt.Sprintf("Data source %d", i), bySource[id])) + groups = append(groups, buildGroup(id, fmt.Sprintf("Data source %d", i), bySource[id], cfg.APIKey)) delete(bySource, id) } // Any result with an unrecognized source is emitted in a trailing group @@ -137,7 +145,7 @@ func BuildDocument(rep validator.Report, cfg config.Config, now time.Time) Docum } sort.Strings(leftover) for _, k := range leftover { - groups = append(groups, buildGroup(k, k, bySource[k])) + groups = append(groups, buildGroup(k, k, bySource[k], cfg.APIKey)) } return Document{ @@ -148,20 +156,19 @@ func BuildDocument(rep validator.Report, cfg config.Config, now time.Time) Docum } } -func buildGroup(id, label string, results []validator.Result) Group { +func buildGroup(id, label string, results []validator.Result, apiKey string) Group { g := Group{ID: id, Label: label, Results: []Item{}} for _, r := range results { cat, step := splitCheck(r.Check) - status := r.Status.String() g.Results = append(g.Results, Item{ Check: r.Check, Category: cat, Step: step, - Status: status, - Message: r.Message, + Status: r.Status.String(), + Message: redactString(r.Message, apiKey), // defense in depth; checks redact upstream Details: r.Details, }) - g.Counts.add(status) + g.Counts.add(r.Status) } return g } @@ -201,7 +208,7 @@ func buildSummary(rep validator.Report, groups []Group) Summary { return Summary{ Verdict: verdict, ExitCode: rep.ExitCode(), - Total: total.Pass + total.Warn + total.Fail + total.Skip, + Total: len(rep.Results), Counts: total, } } diff --git a/report/document_test.go b/report/document_test.go index 4b1b32b..fc87cd0 100644 --- a/report/document_test.go +++ b/report/document_test.go @@ -147,3 +147,90 @@ func TestBuildDocument_RedactsAPIKey(t *testing.T) { t.Errorf("expected redaction marker in output:\n%s", b) } } + +func TestCountsAddPanicsOnUnknownStatus(t *testing.T) { + defer func() { + if recover() == nil { + t.Error("expected panic on out-of-range status") + } + }() + var c Counts + c.add(validator.Status(99)) +} + +func TestBuildDocument_SkipCounted(t *testing.T) { + rep := validator.Report{Results: []validator.Result{ + {Check: "dep", Status: validator.Skip, Message: "prerequisite failed"}, + }} + doc := BuildDocument(rep, config.Config{OBAServerURL: "x"}, fixedTime()) + if doc.Groups[0].Counts.Skip != 1 || doc.Summary.Counts.Skip != 1 { + t.Errorf("skip not counted: group=%+v summary=%+v", doc.Groups[0].Counts, doc.Summary.Counts) + } + if doc.Summary.Total != 1 || doc.Summary.Verdict != "PASS" || doc.Summary.ExitCode != 0 { + t.Errorf("skip-only run: total=%d verdict=%q exit=%d want 1/PASS/0", doc.Summary.Total, doc.Summary.Verdict, doc.Summary.ExitCode) + } +} + +func TestBuildDocument_LeftoverSourceGroup(t *testing.T) { + // Sources that match neither "" nor a configured dataSource[i] must still be + // emitted (sorted) so no data is dropped. + rep := validator.Report{Results: []validator.Result{ + {Check: "a", Status: validator.Pass, Source: "zzz"}, + {Check: "b", Status: validator.Fail, Source: "dataSource[5]"}, + }} + cfg := config.Config{OBAServerURL: "x", DataSources: []config.DataSource{{}}} + doc := BuildDocument(rep, cfg, fixedTime()) + if len(doc.Groups) != 4 { + t.Fatalf("groups=%d want 4 (server, dataSource[0], dataSource[5], zzz)", len(doc.Groups)) + } + if doc.Groups[2].ID != "dataSource[5]" || doc.Groups[3].ID != "zzz" { + t.Errorf("leftover groups not sorted: %q, %q", doc.Groups[2].ID, doc.Groups[3].ID) + } + if doc.Summary.Total != 2 { + t.Errorf("total=%d want 2 (leftover results still counted)", doc.Summary.Total) + } +} + +func TestBuildDocument_DetailsPassthroughAndOmit(t *testing.T) { + doc := BuildDocument(sampleReport(), sampleConfig(), fixedTime()) + it := doc.Groups[1].Results[0] // the vehicle-positions result with Details + if it.Details["vehicleId"] != "1_1234" { + t.Errorf("details not passed through: %+v", it.Details) + } + // A result with nil Details omits the field entirely. + b, _ := json.Marshal(doc.Groups[0].Results[0]) + if strings.Contains(string(b), "details") { + t.Errorf("nil details should be omitted:\n%s", b) + } +} + +func TestBuildDocument_NoDataSources(t *testing.T) { + rep := validator.Report{Results: []validator.Result{{Check: "x", Status: validator.Pass}}} + doc := BuildDocument(rep, config.Config{OBAServerURL: "x"}, fixedTime()) + if len(doc.Groups) != 1 || doc.Groups[0].ID != "server" { + t.Errorf("expected only a server group, got %d groups", len(doc.Groups)) + } + b, _ := json.Marshal(doc.Meta) + if !strings.Contains(string(b), `"dataSources":[]`) { + t.Errorf("empty dataSources must marshal as [] not null:\n%s", b) + } +} + +func TestBuildDocument_Deterministic(t *testing.T) { + a, _ := json.Marshal(BuildDocument(sampleReport(), sampleConfig(), fixedTime())) + b, _ := json.Marshal(BuildDocument(sampleReport(), sampleConfig(), fixedTime())) + if string(a) != string(b) { + t.Errorf("output not deterministic:\n%s\n---\n%s", a, b) + } +} + +func TestBuildDocument_RedactsMessage(t *testing.T) { + rep := validator.Report{Results: []validator.Result{ + {Check: "x", Status: validator.Fail, Message: "failed talking to https://x/?key=SEKRET"}, + }} + cfg := config.Config{OBAServerURL: "https://x", APIKey: "SEKRET"} + doc := BuildDocument(rep, cfg, fixedTime()) + if strings.Contains(doc.Groups[0].Results[0].Message, "SEKRET") { + t.Errorf("apiKey not redacted from message: %q", doc.Groups[0].Results[0].Message) + } +} diff --git a/report/report.go b/report/report.go index c8fa768..a85c27b 100644 --- a/report/report.go +++ b/report/report.go @@ -11,18 +11,27 @@ import ( "github.com/onebusaway/oba-validator/validator" ) -// WriteJSON writes the report as an indented, UI-oriented JSON Document. +// WriteJSON writes the report as an indented, UI-oriented JSON Document. The +// document is marshalled fully before writing so a mid-stream write failure +// can't leave partial, unparseable JSON on the consumer's stream. func WriteJSON(w io.Writer, rep validator.Report, cfg config.Config) error { - enc := json.NewEncoder(w) - enc.SetIndent("", " ") - return enc.Encode(BuildDocument(rep, cfg, time.Now().UTC())) + return writeIndentedJSON(w, BuildDocument(rep, cfg, time.Now().UTC())) } // WriteErrorJSON writes an indented ErrorDocument to w, redacting apiKey from msg. func WriteErrorJSON(w io.Writer, msg, apiKey string) error { - enc := json.NewEncoder(w) - enc.SetIndent("", " ") - return enc.Encode(ErrorDocument{SchemaVersion: SchemaVersion, Error: redactString(msg, apiKey)}) + return writeIndentedJSON(w, ErrorDocument{SchemaVersion: SchemaVersion, Error: redactString(msg, apiKey)}) +} + +// writeIndentedJSON marshals v fully, then writes it in a single call so output +// is all-or-nothing rather than incrementally streamed. +func writeIndentedJSON(w io.Writer, v any) error { + b, err := json.MarshalIndent(v, "", " ") + if err != nil { + return err + } + _, err = w.Write(append(b, '\n')) + return err } // WriteText writes a human-readable, grouped report with a summary line. From c8ee9d2532b352eedd511f243d843b4fa445c16e Mon Sep 17 00:00:00 2001 From: Aaron Brethorst Date: Mon, 25 May 2026 14:24:24 -0700 Subject: [PATCH 10/10] docs: fix invalid JSON example and untagged fenced block Address CodeRabbit review nits on PR #2: the README error-object example was not valid JSON syntax, and the spec's flow-diagram fence lacked a language tag (MD040). --- README.md | 2 +- docs/superpowers/specs/2026-05-25-json-ui-output-design.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 47f642d..2bdf977 100644 --- a/README.md +++ b/README.md @@ -51,7 +51,7 @@ os.Exit(rep.ExitCode()) visualization. It contains `meta` (run inputs — never the apiKey), `summary` (verdict + status counts), and `groups` (a `server` group plus one per data source, each with its results). On failure before a report is produced, a -`{ "schemaVersion", "error" }` object is emitted to stdout and the process exits 2. +an object like `{ "schemaVersion": "1.0", "error": "..." }` is emitted to stdout and the process exits 2. The full contract is published as a JSON Schema (draft 2020-12) at [`schema/oba-validator-report.schema.json`](schema/oba-validator-report.schema.json). diff --git a/docs/superpowers/specs/2026-05-25-json-ui-output-design.md b/docs/superpowers/specs/2026-05-25-json-ui-output-design.md index 4d13010..9f20675 100644 --- a/docs/superpowers/specs/2026-05-25-json-ui-output-design.md +++ b/docs/superpowers/specs/2026-05-25-json-ui-output-design.md @@ -49,7 +49,7 @@ cycle — `config` depends on neither). ### Flow -``` +```text config.Load ──err──▶ (--json? WriteErrorJSON : text) ; exit 2 │ ok validator.Run ──err──▶ (--json? WriteErrorJSON : text) ; exit 2