Skip to content

Add CI integration test suite#34

Open
drdrew42 wants to merge 2 commits intoopenwebwork:developfrom
drdrew42:ci-tests
Open

Add CI integration test suite#34
drdrew42 wants to merge 2 commits intoopenwebwork:developfrom
drdrew42:ci-tests

Conversation

@drdrew42
Copy link
Member

Summary

  • Modernize Docker build: multi-stage build (builder + runtime) cuts image size ~50%, adds .dockerignore, pins Perl deps in cpanfile, updates PG submodule to 2.20.1 (PG main with hotfixes)
  • Add integration test suite: 36 bash+curl+jq tests across 4 categories — smoke, render parity (raw vs JWE/JWS), answer cycle (correct/wrong/partial scoring), and endpoint validation
  • New GitHub Actions workflow (test.yml): builds container, runs PG unit tests (informational), then all integration suites with per-suite CI step visibility

Test categories

  • Smoke (12 tests): health, editor UI, response structure, JWT presence, score defaults, instructor vs student response differences
  • Render parity (8 tests): raw params and explicit JWE/JWS token paths produce identical rendered HTML (normalized to strip non-deterministic attributes)
  • Answer cycle (8 tests): correct → score=1, wrong → score=0, partial → score=0.5, preview mode validation
  • Endpoints (8 tests): static assets, catalog/tap file ops, error handling for missing problems, malformed JWTs, conflicting submit+preview

Running locally

bash t/ci/run-all.sh            # full build + test
bash t/ci/run-all.sh --no-build # skip Docker build (reuse existing image)

Test plan

  • Fresh Docker build from ci-tests branch
  • All 36 integration tests pass
  • PG unit tests run (informational, don't gate CI)
  • Verified render parity normalization distinguishes instructor vs student output

🤖 Generated with Claude Code

drdrew42 and others added 2 commits February 26, 2026 17:06
Multi-stage Dockerfile (Ubuntu 24.04, Node 22) cuts image size from
~1.5GB to ~770MB. Builder stage compiles XS modules and JS assets;
runtime stage copies only what's needed to serve requests.

- Add cpanfile for reproducible Perl dependency installs
- Add .dockerignore to exclude docs, tests, git metadata from context
- Update PG submodule to 2.20.1 (PG-2.20 tag + 21 hotfixes)
- Update pg_config.yml for PG 2.20 macro/context paths
- Rewrite README with configuration reference, deployment topologies,
  and JWT documentation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bash + curl + jq integration tests that run against the Docker container.
Four test suites validate the render API end-to-end:

- Smoke: health check, response structure, instructor vs student fields
- Render parity: raw params vs JWE/JWS JWT produce identical content
- Answer cycle: correct/wrong scoring, partial credit, preview mode
- Endpoints: IO routes, error handling, malformed JWT, simultaneous
  submit+preview rejection

PG unit tests run for visibility but do not gate CI (upstream's
responsibility).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant