ci(github): add container support for self-hosted runners#9
ci(github): add container support for self-hosted runners#9dipankardas011 wants to merge 56 commits intomainfrom
Conversation
3298b6c to
15eae35
Compare
Switch all workflows to use containers on self-hosted runners for improved consistency and isolation. Add system dependency installation steps and configure Docker and Git safe directory where needed. Update job dependencies and permissions for better security and workflow reliability. This enables better compatibility with shared self-hosted environments. chore(ci): use public-repo runner in all workflows Replace the custom self-hosted runner labels with [public-repo] in all GitHub Actions workflow files. This change standardizes the runner environment and improves compatibility for public repositories. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com> fix the core volume mount problem Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com> fix(ci): update wp-env test config and add --update flag - Remove "plugins" field from .wp-env.test.json to prevent broken mounts - Add dynamic host mapping for plugin in test config - Add --update flag to npm run wp-env:test start commands for fresh env Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com> ci: improve Docker mapping and plugin activation Update workflows to fix wp-env host mapping, add plugin activation steps, and ensure Docker environments are properly started and stopped. Switch PHPUnit commands to use tests-cli for accurate environment handling. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
15eae35 to
5e132e8
Compare
Refactor GitHub Actions workflows to standardize environment variable usage and port assignments for wp-env across build, E2E, and PHPUnit jobs. Removes dynamic COMPOSE_PROJECT_NAME generation and sets static WP_ENV_PORT and WP_ENV_TESTS_PORT values to avoid conflicts on shared self-hosted runners. Updates .wp-env.test.json to include testsPort. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
f8d1bec to
9289434
Compare
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
9289434 to
872e914
Compare
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Ensure wp-env uses a dedicated /tmp/wp-env directory for all jobs by setting WP_ENV_HOME and mounting it in the container. This prevents permission issues and data conflicts in CI. Updates all workflow steps to use the new environment variable. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
- Increase `timeout-minutes` for build, e2e, and phpunit workflows to 30 minutes for more reliable CI runs. - Set `timeout_minutes` to 15 for Docker environment startup steps. - Add `git config --global --add safe.directory '/tmp/wp-env/*'` to address Git safe directory issues in Docker. - Run E2E tests with `--debug` for better diagnostics. - No functional changes to plugin code. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Update CI workflow to use step outputs for job filtering instead of hardcoded values. This enables more flexible and accurate job execution based on file changes. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
When running in the Docker-out-of-Docker CI setup, the actions/checkout step creates the workspace files as the `root` user. However, wp-env dynamically configures its internal test containers to run as `wpuser` (UID 1001) to match the host user executing `wp-env start`. This caused a permission collision where the test runner (running as UID 1001) could not write test artifacts (like .phpunit.result.cache and code coverage XML/HTML reports) back to the workspace directory owned by root. This adds a step to `chown -R wpuser:wpuser .` across the build, e2e, and phpunit workflows immediately before starting wp-env. This ensures the containerized test runner has the necessary write privileges to generate and upload coverage reports successfully. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
857f1e8 to
ac888b5
Compare
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Add --add-host=host.docker.internal:host-gateway to container options in the reusable-e2e.yml workflow. Update WP_BASE_URL to use host.docker.internal for E2E tests, enabling proper connectivity between Playwright and the WordPress instance when running in Docker. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Use --network=host for the CI container to ensure Playwright shares the host Docker daemon's network. This prevents "ECONNREFUSED" errors when WordPress issues 302 redirects to "localhost", avoiding loopback traps inside the CI container. Also remove WP_BASE_URL env from test command. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Switch CI container from --network=host to --add-host and socat proxy for localhost:8889. This ensures Playwright and WordPress agree on "localhost" and prevents network isolation issues. Adds socat to dependencies and updates E2E test step to forward traffic, improving test stability. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Set Playwright to run in headless mode by default for more consistent and reliable test execution in CI and local environments. This avoids issues with UI rendering and ensures tests do not require a display server. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Removed the --debug flag from the E2E test command in the CI workflow to ensure tests run in standard mode. Updated Playwright config to spread baseConfig.use for better configuration inheritance. These changes improve consistency and maintainability of E2E test execution. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Updated CI workflow to use dynamic outputs from filter steps instead of hardcoded values. Enabled e2e job to trigger based on detection results and phpunit status. Removed outdated comments for clarity. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
|
@dipankardas011 is there a way to cache those install commands? Setting up docker, php, reinstalling npm deps... (not composer, since that's handled by our action) |
| contents: read | ||
|
|
||
| env: | ||
| WP_ENV_PORT: 8889 |
There was a problem hiding this comment.
I was able to get rid of the (hopefully) unnecessary WP_TESTS_ENV_PORT everywhere when I fixed the .tests.wp-env.json file . I'm pretty sure we can get rid of this one too but the CI is waaay to slow and I ran out of time to try. What do you think?
As a reminder, we're using wp-env 11.1. so there's 1 encironment in a JSON and it defaults to the prt defined in the JSON or the constant
|
@dipankardas011 I've restored the public gh runners to live alongside these private one, and started on some cleanup regarding the I ran out of time to finish testing because of how slow things are, so please review and finish up what else is needed. As you can see both PHPUnit and Playwright E2E tests are passing on the GH runners, so they should all be passing on the private runner side too. 🙇 Edit: and they are 🚀 So just playground and cleanup/optim left. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9 +/- ##
=========================================
Coverage 94.94% 94.94%
Complexity 115 115
=========================================
Files 21 21
Lines 475 475
=========================================
Hits 451 451
Misses 24 24
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
3736ad7 to
5c37a7f
Compare
|
@justlevine I saw your recent changes to the Public workflows which you created Seems like workflow is working, Not sure why the composer.yaml Refer is not found may be the changes from the main branch had a problem it seems I guess. |
So what I can see is we have a single self-hosted runner and when job run happens it only assign one at a time. this is where we are getting the Major slowdown. and then comes up is the setup of the workload like adding system deps Yes ~7m each but then comes another point that not all of them needs to be tested thanks to |
As we are running inside docker and the only way (provided we are READY to maintain a separate OCI image in this repo's ghcr.io registry and same for the workflow file, Point is do we need that? Let me know) as you pointed out yes ~30minutes the only steps we can getinside the container are
If you check how much in total of the workflow runtime it is and whether its worth it or not. |
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
|
@justlevine I came across a different approach of using the github's official ubuntu24.04 image I tried to change only the workflow But it seems like the composer is failing I don't know what is the problem anyone who knows that? after that is fixed I can then check if this modified build (private) one is working then I will update the rest of the private workflows ;) I refered to this https://github.com/actions/runner-images/releases/tag/ubuntu24%2F20260309.50 Caution for now I did used a tag some 2.33 something which when ran it for |
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Enhance post-run cleanup steps in all reusable GitHub Actions workflows to remove artifacts from both /__w and $HOME directories. This prevents pollution between workflow runs and ensures a cleaner environment for subsequent jobs. Updates the cleanup step name for clarity. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Enhances the post-run cleanup steps in reusable GitHub Actions workflows. Now uses `sudo` to ensure all files, including hidden ones, are removed from the workspace, home, and workflow directories. Also selectively deletes contents in /__w/_temp/ while preserving locked mount points. This prevents artifact and environment pollution between workflow runs. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Switch Jest and JS lint workflows to use the ghcr.io/actions/actions-runner:2.333.0 container image instead of node:22. This aligns the CI environment with GitHub Actions runner standards and may improve compatibility and consistency across jobs. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Add steps to reusable GitHub Actions workflows to remove contents of /tmp/wp-env in build, e2e, and phpunit jobs, and /tmp/playwright-cache in e2e jobs. This ensures a clean environment for each workflow run and prevents leftover files from affecting subsequent jobs. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>

What
Switch all workflows to use containers on self-hosted runners for improved
consistency and isolation. Add system dependency installation steps and
configure Docker and Git safe directory where needed. Update job dependencies
and permissions for better security and workflow reliability. This enables
better compatibility with shared self-hosted environments.