fix: re-enabled e2e tests for orchestrator workflows. #3966

lokanandaprabhu · 2026-01-07T09:12:20Z

Description

In PR #3810, the Orchestrator plugins e2e tests were disabled because there were no Backstage 1.45 compatible OCI images available at the time, which caused the backend to fail during startup.

The Backstage 1.45.3 compatible OCI images are now available in the GitHub Container Registry:

bs_1.45.3__5.1.0 - orchestrator frontend
bs_1.45.3__8.3.0 - orchestrator backend
bs_1.45.3__1.1.0 - scaffolder-backend-module-orchestrator
bs_1.45.3__1.2.0 - orchestrator-form-widgets

Changes

Restored Orchestrator Plugins

.ibm/pipelines/value_files/values_showcase.yaml - Added orchestrator plugins with 1.45.3 versions
.ibm/pipelines/value_files/values_showcase-rbac.yaml - Added orchestrator plugins with 1.45.3 versions

Re-enabled E2E Tests

e2e-tests/playwright.config.ts - Removed global testIgnore pattern for orchestrator tests
e2e-tests/playwright/e2e/plugins/orchestrator/workflow-all-runs-validations.spec.ts - Removed test.skip(true, "Disabled on Backstage 1.45 update")

Which issue(s) does this PR fix

Fixes #?

https://issues.redhat.com/browse/RHIDP-11262

Please make sure that the following steps are complete:

GitHub Actions are completed and successful
Unit Tests are updated and passing
E2E Tests are updated and passing
Documentation is updated if necessary (requirement for new features)
Add a screenshot if the change is UX/UI related

rhdh-qodo-merge · 2026-01-07T09:12:46Z

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 RHIDP-11262 - Partially compliant Compliant requirements: Restore `.ibm/pipelines/value_files/values_showcase-rbac.yaml` Restore `.ibm/pipelines/value_files/values_showcase.yaml` Restore `e2e-tests/playwright/e2e/plugins/orchestrator/workflow-all-runs-validations.spec.ts` Restore `e2e-tests/playwright.config.ts` Re-enable Orchestrator plugin e2e tests Update Orchestrator plugin to a Backstage 1.45-compatible version (via 1.45.3-tagged OCI packages) Non-compliant requirements: Requires further human verification: Confirm the referenced 1.45.3 OCI images exist, are accessible from CI, and contain the expected plugin builds. Confirm e2e orchestrator tests pass in the intended CI environments and the skips (OSD_GCP, GKE) are still correct. Confirm the restored Helm values work end-to-end (backend starts, orchestrator routes render, workflows tab mounts, Data Index URL is correct in the target cluster).
⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🔒 No security concerns identified
⚡ Recommended focus areas for review Config drift The orchestrator plugin configuration is duplicated across multiple values files; ensure the `dynamicPlugins.frontend` keys, mount points, and plugin IDs match the expected plugin manifests for Backstage 1.45.3 and remain consistent with the RBAC variant to avoid subtle runtime differences. # Enable orchestrator plugins - Official release (Backstage 1.45.3) - package: "oci://ghcr.io/redhat-developer/rhdh-plugin-export-overlays/red-hat-developer-hub-backstage-plugin-orchestrator:bs_1.45.3__5.1.0!red-hat-developer-hub-backstage-plugin-orchestrator" disabled: false pluginConfig: dynamicPlugins: frontend: red-hat-developer-hub.backstage-plugin-orchestrator: appIcons: - importName: OrchestratorIcon name: orchestratorIcon dynamicRoutes: - importName: OrchestratorPage menuItem: icon: orchestratorIcon text: Orchestrator textKey: menuItem.orchestrator path: /orchestrator entityTabs: - path: /workflows title: Workflows titleKey: catalog.entityPage.workflows.title mountPoint: entity.page.workflows mountPoints: - mountPoint: entity.page.workflows/cards importName: OrchestratorCatalogTab config: layout: gridColumn: "1 / -1" if: anyOf: - IsOrchestratorCatalogTabAvailable - package: "oci://ghcr.io/redhat-developer/rhdh-plugin-export-overlays/red-hat-developer-hub-backstage-plugin-orchestrator-backend:bs_1.45.3__8.3.0!red-hat-developer-hub-backstage-plugin-orchestrator-backend" disabled: false pluginConfig: orchestrator: dataIndexService: url: http://sonataflow-platform-data-index-service - package: "oci://ghcr.io/redhat-developer/rhdh-plugin-export-overlays/red-hat-developer-hub-backstage-plugin-scaffolder-backend-module-orchestrator:bs_1.45.3__1.1.0!red-hat-developer-hub-backstage-plugin-scaffolder-backend-module-orchestrator" disabled: false pluginConfig: orchestrator: dataIndexService: url: http://sonataflow-platform-data-index-service - package: "oci://ghcr.io/redhat-developer/rhdh-plugin-export-overlays/red-hat-developer-hub-backstage-plugin-orchestrator-form-widgets:bs_1.45.3__1.2.0!red-hat-developer-hub-backstage-plugin-orchestrator-form-widgets" disabled: false pluginConfig: dynamicPlugins: frontend: red-hat-developer-hub.backstage-plugin-orchestrator-form-widgets: {} Service URL The `orchestrator.dataIndexService.url` is hardcoded to an in-cluster service name; verify this is correct for all target environments and that the service is deployed/available where these values are used, otherwise orchestrator backend and related e2e tests may fail at runtime. - package: "oci://ghcr.io/redhat-developer/rhdh-plugin-export-overlays/red-hat-developer-hub-backstage-plugin-orchestrator-backend:bs_1.45.3__8.3.0!red-hat-developer-hub-backstage-plugin-orchestrator-backend" disabled: false pluginConfig: orchestrator: dataIndexService: url: http://sonataflow-platform-data-index-service - package: "oci://ghcr.io/redhat-developer/rhdh-plugin-export-overlays/red-hat-developer-hub-backstage-plugin-scaffolder-backend-module-orchestrator:bs_1.45.3__1.1.0!red-hat-developer-hub-backstage-plugin-scaffolder-backend-module-orchestrator" disabled: false pluginConfig: orchestrator: dataIndexService: url: http://sonataflow-platform-data-index-service CI stability Removing the global ignore will re-enable all orchestrator specs; validate overall CI runtime/flakiness and confirm job matrix still provides the required infra (orchestrator backend + Data Index) for the environments where tests will now execute. export default defineConfig({ timeout: 90 * 1000, testDir: "./playwright", /* Fail the build on CI if you accidentally left test.only in the source code. / forbidOnly: !!process.env.CI, / Retry on CI only / retries: process.env.CI ? 2 : 0, / Opt out of parallel tests on CI. */ workers: 3,
📄 References No matching references available

rhdh-qodo-merge · 2026-01-07T09:12:55Z

PR Type

Bug fix, Tests

Description

Re-enabled orchestrator e2e tests with Backstage 1.45.3 compatible OCI images
Restored orchestrator plugin configurations in showcase value files
Removed global test ignore pattern blocking orchestrator tests
Removed skip directive from orchestrator workflow validation tests

File Walkthrough

Relevant files

Tests

playwright.config.ts `Remove orchestrator test ignore pattern` e2e-tests/playwright.config.ts Removed global `testIgnore` pattern that was skipping all orchestrator test files Allows orchestrator e2e tests to run as part of the test suite	+0/-4
workflow-all-runs-validations.spec.ts `Remove skip directive from workflow tests` e2e-tests/playwright/e2e/plugins/orchestrator/workflow-all-runs-validations.spec.ts Removed `test.skip(true, "Disabled on Backstage 1.45 update")` directive Keeps conditional skips for OSD-GCP and GKE environments Enables orchestrator workflow validation tests to execute	+0/-1

Configuration changes

values_showcase.yaml `Restore orchestrator plugins with 1.45.3 versions` .ibm/pipelines/value_files/values_showcase.yaml Added four orchestrator plugin packages with Backstage 1.45.3 versions Configured orchestrator frontend plugin with routes, icons, and entity tabs Configured orchestrator backend plugin with data index service URL Configured scaffolder backend module and form widgets for orchestrator	+51/-1
values_showcase-rbac.yaml `Restore orchestrator plugins with 1.45.3 versions` .ibm/pipelines/value_files/values_showcase-rbac.yaml Added four orchestrator plugin packages with Backstage 1.45.3 versions Configured orchestrator frontend plugin with routes, icons, and entity tabs Configured orchestrator backend plugin with data index service URL Configured scaffolder backend module and form widgets for orchestrator	+51/-1

rhdh-qodo-merge · 2026-01-07T09:13:56Z

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Security	Use HTTPS for service URL Change the `dataIndexService` URL from `http` to `https` to ensure secure, encrypted data transmission. .ibm/pipelines/value_files/values_showcase-rbac.yaml [165-167] orchestrator: dataIndexService: - url: http://sonataflow-platform-data-index-service + url: https://sonataflow-platform-data-index-service Apply / Chat Suggestion importance[1-10]: 8 __ Why: The suggestion correctly identifies a security risk by pointing out the use of an unencrypted `http` endpoint for service communication and recommends switching to `https` for improved security.	Medium
General	Increase test timeout Increase the global test timeout in `playwright.config.ts` to prevent flakiness from longer-running orchestrator tests. e2e-tests/playwright.config.ts [25-26] export default defineConfig({ - timeout: 90 * 1000, + timeout: 120 * 1000, `[To ensure code accuracy, apply this suggestion manually]` Suggestion importance[1-10]: 6 __ Why: The suggestion provides a reasonable performance improvement by increasing the test timeout, which could help prevent flaky tests now that orchestrator tests are re-enabled.	Low
Update

github-actions · 2026-01-07T10:08:07Z

The image is available at:

/test e2e-ocp-helm

lokanandaprabhu · 2026-01-07T11:18:18Z

/retest

christoph-jerolimov

/cc @christoph-jerolimov @hopehadfield
/lgtm
/approve

lokanandaprabhu · 2026-01-07T12:59:28Z

/retest

…error

github-actions · 2026-01-07T13:25:09Z

🚫 Image Push Skipped.

The container image push was skipped because the build was skipped (either due to [skip-build] tag or no relevant changes with existing image)

/test e2e-ocp-helm

sonarqubecloud · 2026-01-07T13:25:27Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

christoph-jerolimov

Thanks again 👍

e2e passed

/lgtm
/approve

openshift-ci · 2026-01-09T09:54:07Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: christoph-jerolimov

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~.ibm/OWNERS~~ [christoph-jerolimov]
~~e2e-tests/OWNERS~~ [christoph-jerolimov]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

christoph-jerolimov · 2026-01-09T09:58:42Z

Skip retest!

fix: Re-enabled e2e tests for orchestrator workflows.

1207fa7

openshift-ci bot requested review from albarbaro and psrna January 7, 2026 09:12

lokanandaprabhu temporarily deployed to internal January 7, 2026 09:12 — with GitHub Actions Inactive

rhdh-qodo-merge bot added Review effort 2/5 Tests Bug fix labels Jan 7, 2026

lokanandaprabhu changed the title ~~fix: Re-enabled e2e tests for orchestrator workflows.~~ fix: re-enabled e2e tests for orchestrator workflows. Jan 7, 2026

christoph-jerolimov approved these changes Jan 7, 2026

View reviewed changes

openshift-ci bot requested review from christoph-jerolimov and hopehadfield January 7, 2026 11:25

openshift-ci bot assigned christoph-jerolimov Jan 7, 2026

openshift-ci bot added lgtm approved labels Jan 7, 2026

fix: disable scaffolder-backend-module-orchestrator - causes startup …

135eff0

…error

openshift-ci bot removed the lgtm label Jan 7, 2026

lokanandaprabhu temporarily deployed to internal January 7, 2026 13:24 — with GitHub Actions Inactive

christoph-jerolimov approved these changes Jan 9, 2026

View reviewed changes

openshift-ci bot added the lgtm label Jan 9, 2026

christoph-jerolimov merged commit d65cd35 into redhat-developer:main Jan 9, 2026
18 of 19 checks passed

fix: re-enabled e2e tests for orchestrator workflows. #3966

fix: re-enabled e2e tests for orchestrator workflows. #3966

Conversation

lokanandaprabhu commented Jan 7, 2026

Description

Changes

Restored Orchestrator Plugins

Re-enabled E2E Tests

Which issue(s) does this PR fix

Uh oh!

rhdh-qodo-merge bot commented Jan 7, 2026

PR Reviewer Guide 🔍

Uh oh!

rhdh-qodo-merge bot commented Jan 7, 2026

PR Type

Description

File Walkthrough

Uh oh!

rhdh-qodo-merge bot commented Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Code Suggestions ✨

Uh oh!

github-actions bot commented Jan 7, 2026

Uh oh!

lokanandaprabhu commented Jan 7, 2026

Uh oh!

christoph-jerolimov left a comment

Choose a reason for hiding this comment

Uh oh!

lokanandaprabhu commented Jan 7, 2026

Uh oh!

github-actions bot commented Jan 7, 2026

Uh oh!

sonarqubecloud bot commented Jan 7, 2026

Quality Gate passed

Uh oh!

christoph-jerolimov left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

openshift-ci bot commented Jan 9, 2026

Uh oh!

christoph-jerolimov commented Jan 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

rhdh-qodo-merge bot commented Jan 7, 2026 •

edited

Loading

christoph-jerolimov left a comment •

edited

Loading