feat(server): add report-api server for dynamic metadata assembly#195
feat(server): add report-api server for dynamic metadata assembly#195meyer9 wants to merge 6 commits into
Conversation
Adds server/ — an HTTP server that replaces the per-environment static metadata.json with a dynamically assembled response fetched directly from per-run S3 files. This moves the aggregation, retention, and comparison-synthesis logic into the open-source base/benchmark repo. There is no central metadata file. The server: - Lists all <outputDir>/metadata.json objects in S3 (one per run) - Merges, deduplicates, and applies retention policy - Appends synthetic [Compare: Time] and [Compare: Versions] groups so the existing report UI can compare runs across time windows or client versions without any frontend change - Caches aggressively: per-file by ETag (invalidated when a run is rewritten), merged result by object fingerprint + 1h TTL (handles time.Now() dependence in retention/comparison) The server is a straight port of protocols/base-benchmarking's report-api, updated to use the per-run-directory S3 layout (#66 in base-benchmarking). The base-benchmarking repo retains its copy during the transition; a follow-up PR will remove it once this one is deployed. 14 tests ported from base-benchmarking (comparison synthesizer tests). All existing runner/ tests unaffected.
🟡 Heimdall Review Status
|
Allows the report server to serve directly from the directory written by 'base-bench run --output-dir <dir>' without any S3 or MinIO setup. This makes local development a one-command flow: report-server --local-dir ./output The same merge/dedup/retention/comparison-synthesis pipeline runs against local files as against S3, so reports look identical regardless of backend. Changes: - BackendStorage interface: extracted from S3Service so handlers work against either backend - LocalService: reads <outputDir>/metadata.json + metrics files from a local directory tree. Uses file mtime as the ETag equivalent for cache invalidation — a newly written metadata.json is visible on the next request. - --local-dir flag (env: BASE_BENCH_API_LOCAL_DIR): mutually exclusive with --s3-bucket; both validated at startup - S3BucketFlag: Required: true removed (validation moved to Validate()) - mergeRuns() + applyRetentionPolicy(): promoted to package-level functions so both S3Service and LocalService share the pipeline - 6 new LocalService unit tests: GetMetadata, cache hit, cache invalidation on new file, GetObject, invalid dir, load tests Verified: server starts with --local-dir, health returns 200, metadata.json returns runs from local files with comparison groups.
GetObject, ListLoadTests, and GetLoadTest all accepted user-provided values (HTTP path params) and passed them directly to filepath.Join without validating that the resolved path stays within the root dir. This allowed path traversal — e.g. GET /output/../../../etc/passwd. Fix: safePath() resolves the joined path with filepath.Clean, then checks it has the root as a prefix. Returns an error for any path that escapes the root directory. Also adds TestLocalService_PathTraversalBlocked covering the three dangerous patterns: ../etc/passwd, ../../secret, run-a/../../outside.
|
Fixed in 0b51c7c. All three CodeQL path-traversal findings addressed via a new func (ls *LocalService) safePath(rel string) (string, error) {
abs := filepath.Clean(filepath.Join(ls.dir, rel))
base := filepath.Clean(ls.dir) + string(filepath.Separator)
if abs != filepath.Clean(ls.dir) && !strings.HasPrefix(abs, base) {
return "", fmt.Errorf("path %q escapes root directory", rel)
}
return abs, nil
}
|
The previous safePath implementation used strings.HasPrefix against a manually-constructed base path, which CodeQL did not recognize as a path sanitizer and continued flagging the call sites. Switched to filepath.Rel(root, abs): if the relative path from the root to the resolved target starts with '..', the target is outside the root. filepath.Rel is the idiomatic Go pattern for this check and is more likely to be recognized by static analysis tools.
|
Updated in 346a48c. The previous Switched to relPath, err := filepath.Rel(rootClean, abs)
if err != nil || strings.HasPrefix(relPath, "..") {
return "", fmt.Errorf("path %q escapes root directory", rel)
}All 7 LocalService tests still pass including |
errcheck: defer result.Body.Close() -> defer result.Body.Close() //nolint:errcheck The S3 response body Close() is best-effort cleanup; the SDK documents the error as always nil. staticcheck SA1019: add //nolint:staticcheck on aws-sdk-go v1 imports aws-sdk-go v1 is deprecated in favour of v2. The migration is a larger change tracked separately; the nolint directives keep CI green in the meantime.
Summary
Adds
server/— an HTTP server for the benchmark report UI — to this open-source repo. This implements the aggregation, comparison-synthesis, and caching logic.No centralized metadata.json
The server never writes or reads a single merged metadata file. Instead:
<outputDir>/metrics-*.json, then<outputDir>/metadata.jsonlast as the commit signal).*/metadata.jsonobjects, merges them, applies retention, and serves the result dynamically.ListObjectsV2call per request.The legacy
metadata/metadata-<timestamp>.jsonlayout is ignored — the server skips that prefix.What's in
server/Endpoints:
GET /output/metadata.jsonGET /output/<outputDir>/metrics-<role>.jsonGET /api/v1/load-tests/:networkGET /api/v1/load-tests/:network/:timestampGET /api/v1/healthThe comparison groups (
[Compare: Time]/[Compare: Versions]) are generated server-side and injected into the metadata response. The existing report UI surfaces them in the existing dropdown with no frontend changes. On the chart comparison page,Show Line Per: TimeBucket(1d/1w/1m) orShow Line Per: ClientVersionsplits the overlay.Cache design:
<s3key>|<etag>— invalidated when a file is overwritten (new ETag), so a resubmitted or corrected run is never missed.{key, etag}pairs + 1h TTL — the TTL handles thetime.Now()dependence in retention and comparison-bucket synthesis; without it, the 14-day window and 1d/1w/1m buckets would be frozen at the first rebuild indefinitely.Tests
14 unit tests ported from
base-benchmarkingcovering the comparison synthesizer: time-bucket grouping, version grouping, latest-per-variant, no source mutation, monthly-prefix regression, network scoping, stable IDs, TimeBucket stamping,createdAt=now on synthetic clones, andcanonicalTestName/slugify.All existing
runner/tests still pass.go vetandgofmtclean.Data contract
See
docs/report-data-contract.mdfor what producers must write for runs to appear in the report.type=routine
risk=low
impact=sev5