Skip to content

Conversation

@dhellmann
Copy link
Member

@dhellmann dhellmann commented Dec 17, 2025

Summary

Consolidate e2e tests to reduce CI job count and expand Python version support.

  • Reduced CI jobs by ~85%: From ~105 to ~30 jobs per PR
  • Preserved developer workflow: All individual test scripts remain runnable

Changes

E2E Test Consolidation

  • Created 6 CI suite scripts that run groups of related e2e tests sequentially
  • Added shared framework (ci_suite_framework.sh) for common functionality
  • Enhanced mergify_lint.py to validate all e2e tests are covered by CI suites

Benefits

  • Faster CI: Significantly fewer parallel jobs improve build times
  • Better maintainability: Shared framework eliminates code duplication
  • Future-proof validation: Automatic detection prevents missing test coverage

Test Coverage

All 35 individual e2e tests preserved and validated across 2 Python versions with maintained debugging capabilities.

🤖 Generated with Claude Code

Fixes #882

Co-Authored-By: Claude noreply@anthropic.com

@dhellmann dhellmann requested a review from a team as a code owner December 17, 2025 20:39
@dhellmann dhellmann changed the title feat(ci): consolidate e2e tests into 6 CI suite scripts feat(ci): consolidate e2e tests and expand tests to include python 3.13 and 3.14 Dec 17, 2025
@dhellmann dhellmann marked this pull request as draft December 17, 2025 22:01
@dhellmann dhellmann force-pushed the consolidate-e2e-jobs branch from 8d6bd0c to 10f8d4e Compare December 17, 2025 22:08
@dhellmann dhellmann changed the title feat(ci): consolidate e2e tests and expand tests to include python 3.13 and 3.14 feat(ci): consolidate e2e tests Dec 17, 2025
@dhellmann dhellmann marked this pull request as ready for review December 17, 2025 22:31
@iangelak
Copy link
Contributor

In general everything looks good to me! Should the commits be squashed into one?

@dhellmann
Copy link
Member Author

In general everything looks good to me! Should the commits be squashed into one?

I try to keep each commit logically distinct. That way if we need to revert part of the change in the PR, we don't necessarily have to revert everything.

dhellmann and others added 6 commits December 18, 2025 14:38
Addresses issue python-wheel-build#882 by reducing CI job count from ~105 to ~18 jobs
while maintaining all test coverage and developer workflows.

Changes:
- Create 6 CI suite scripts that run groups of related e2e tests
- Add shared framework (ci_suite_framework.sh) for common functionality
- Update GitHub Actions matrix to use CI suites instead of 35 individual tests
- Update Mergify config to require new consolidated CI job checks
- Enhance mergify_lint.py to validate all e2e tests are covered by CI suites

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
We have a matrix of jobs, let's make sure we force the python
interpreter to be the one we want in case there are multiple installed
in the test image.
Clean e2e-output directory and hatch environment before each test in CI suites
to prevent contamination between tests, particularly from plugin installations
like in test_override.sh that were causing subsequent test failures.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Enhance agent instruction import placement guidelines to be more explicit:
- Add "ALWAYS FOLLOW" to section heading
- Strengthen language: "MUST" instead of "should"
- Add "ALL Python files" clarification
- Emphasize "under any circumstances" for local imports

Ensures future AI agents clearly understand PEP 8 import requirements.

Co-Authored-By: Claude <claude@anthropic.com>
@dhellmann dhellmann force-pushed the consolidate-e2e-jobs branch from 10f8d4e to 45d074d Compare December 18, 2025 19:39
Avoid using emoji in log messages.
@LalatenduMohanty LalatenduMohanty merged commit 9282c74 into python-wheel-build:main Dec 19, 2025
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

consolidate some of the e2e tests

3 participants