Skip to content

Stage 6 backend: canonical report identity and run metadata#3500

Draft
anth-volk wants to merge 15 commits into
masterfrom
feat/report-output-run-stage-6-backend
Draft

Stage 6 backend: canonical report identity and run metadata#3500
anth-volk wants to merge 15 commits into
masterfrom
feat/report-output-run-stage-6-backend

Conversation

@anth-volk
Copy link
Copy Markdown
Collaborator

@anth-volk anth-volk commented Apr 23, 2026

Fixes #3499

Summary

  • make explicit report definitions canonical on API v1
  • create/reuse reports by canonical report identity instead of the old thin sim/year/current-version rule
  • resolve report reads through canonical parents plus display-run selection
  • preserve explicit report specs across later PATCH/sync traffic
  • preserve execution metadata from existing runs instead of rebuilding it from mutable parent rows
  • validate legacy ID mappings by canonical report identity
  • add Alembic scaffolding and a generated migration for the report identity/run schema
  • add coverage for malformed legacy compatibility reads and identity/run-metadata behavior

Rollout

  • Manual Alembic work is required before this PR is merged/deployed to production.
  • First stamp the existing production schema at 60d38593ddc3, then upgrade to 558935decda5.
  • After the schema exists, run the updated Stage 4 data migration/backfill before deploying this code. Stage 4 must preserve exposed historical duplicate report IDs while populating canonical identity/run metadata and legacy ID mappings where applicable.
  • The report identity index is intentionally non-unique so historical duplicate report_outputs rows can remain accessible during and after migration. New normal create traffic is still routed through canonical identity lookup and should reuse an existing report row plus create/reuse run records rather than minting new duplicate reports.
  • Do not deploy this code against a database that lacks legacy_report_output_id_map and the report identity columns/index.

Notes

  • historical identity and alias backfill is handled separately on the updated stage 4 script branch
  • users continue to access run data through the report GET endpoint; direct run browsing/selection remains out of scope

Verification

  • FLASK_DEBUG=1 uv run pytest tests/unit/data/test_run_schema.py tests/unit/services/test_report_output_id_map_service.py tests/unit/test_stage5_routes.py::test_get_report_output_legacy_id_wrong_country_returns_not_found tests/unit/test_stage5_routes.py::test_patch_simulation_explicit_run_id_response_uses_that_run
  • uv run ruff check alembic/versions/20260511_558935decda5_add_report_run_canonical_schema.py policyengine_api/data/alembic_metadata.py policyengine_api/services/report_output_id_map_service.py policyengine_api/services/simulation_service.py policyengine_api/routes/simulation_routes.py tests/unit/data/test_run_schema.py tests/unit/services/test_report_output_id_map_service.py tests/unit/test_stage5_routes.py

@anth-volk anth-volk force-pushed the feat/report-output-run-stage-6-backend branch from f89400d to 11fcff3 Compare May 8, 2026 00:26
@codecov
Copy link
Copy Markdown

codecov Bot commented May 11, 2026

Codecov Report

❌ Patch coverage is 79.19162% with 139 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.46%. Comparing base (1fd9618) to head (94228b3).
⚠️ Report is 6 commits behind head on master.

Files with missing lines Patch % Lines
policyengine_api/services/report_output_service.py 82.38% 31 Missing and 25 partials ⚠️
policyengine_api/data/alembic_metadata.py 0.00% 25 Missing ⚠️
policyengine_api/services/simulation_service.py 80.00% 11 Missing and 14 partials ⚠️
policyengine_api/routes/report_output_routes.py 68.62% 9 Missing and 7 partials ⚠️
policyengine_api/routes/simulation_routes.py 80.00% 2 Missing and 2 partials ⚠️
...ngine_api/services/report_output_id_map_service.py 93.44% 2 Missing and 2 partials ⚠️
policyengine_api/services/report_run_service.py 84.21% 1 Missing and 2 partials ⚠️
policyengine_api/services/report_spec_service.py 91.42% 1 Missing and 2 partials ⚠️
...olicyengine_api/services/simulation_run_service.py 78.57% 1 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3500      +/-   ##
==========================================
- Coverage   77.81%   77.46%   -0.36%     
==========================================
  Files          65       66       +1     
  Lines        3877     4402     +525     
  Branches      697      807     +110     
==========================================
+ Hits         3017     3410     +393     
- Misses        683      772      +89     
- Partials      177      220      +43     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stage 6: make explicit report definitions and run metadata canonical

1 participant