Skip to content

chore(e2e): Add Makefile to make running specific e2e test apps easier #18953

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
andreiborza merged 1 commit into from
Jan 23, 2026
Merged

chore(e2e): Add Makefile to make running specific e2e test apps easier #18953

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions dev-packages/e2e-tests/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
.PHONY: run list

run:
@if ! command -v fzf &> /dev/null; then \
Copy link

@cursor cursor bot Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bash-specific redirection syntax breaks on POSIX shells

High Severity

The &> redirection syntax is bash-specific and not POSIX-compliant. Since the Makefile doesn't specify SHELL := /bin/bash, it defaults to /bin/sh which is dash on many Linux systems including Ubuntu. This causes a syntax error preventing the fzf check from working on non-bash systems. The POSIX-compliant alternative is >/dev/null 2>&1.

Fix in Cursor Fix in Web

echo "Error: fzf is required. Install with: brew install fzf"; \
exit 1; \
fi
@ls test-applications | fzf --height=10 --layout=reverse --border=rounded --margin=1.5% --color=dark --prompt="yarn test:run " | xargs -r yarn test:run
Copy link

@sentry sentry bot Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The Makefile uses xargs -r, which is a GNU-specific flag. This will cause the command to fail on macOS, which uses BSD xargs by default.
Severity: MEDIUM

Suggested Fix

Remove the -r flag from the xargs command. The default behavior of BSD xargs (used on macOS) is to not run the command if the input is empty, which achieves the same goal. This change will make the command compatible with both GNU and BSD systems.

Prompt for AI Agent 
Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.

Location: dev-packages/e2e-tests/Makefile#L8

Potential issue: The `make run` command in `dev-packages/e2e-tests/Makefile` pipes
output to `xargs -r`. The `-r` flag is a GNU extension and is not supported by the BSD
version of `xargs` that is the default on macOS. Since the project's documentation
explicitly instructs macOS users to install dependencies using Homebrew, it is a primary
target development platform. Consequently, any developer on macOS attempting to use this
documented command will encounter an `xargs: illegal option -- r` error, preventing the
test runner from executing.

Did we get this right? 👍 / 👎 to inform future reviews.

Copy link
Member Author

@andreiborza andreiborza Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, it runs on mac.

Copy link

@cursor cursor bot Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

xargs -r flag incompatible with macOS

High Severity

The xargs -r flag is a GNU extension not available in BSD xargs on macOS. Since the documentation instructs users to install fzf via brew (macOS-specific), the Makefile will fail on macOS with error xargs: illegal option -- r. This prevents the entire feature from working on its primary target platform.

Fix in Cursor Fix in Web


list:
@ls test-applications
24 changes: 24 additions & 0 deletions dev-packages/e2e-tests/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,30 @@ yarn test:run <app-name> --variant <variant-name>

Variant name matching is case-insensitive and partial. For example, `--variant 13` will match `nextjs-pages-dir (next@13)` if a matching variant is present in the test app's `package.json`.

### Using the Makefile

Alternatively, you can use the provided Makefile for an interactive test selection experience:

**Prerequisites**: Install `fzf` with Homebrew:

```bash
brew install fzf
```

**Run tests interactively**:

```bash
make run
```

This will display a fuzzy-finder menu of all available test applications. Select one to run it automatically.

**List all test applications**:

```bash
make list
```

For example, if you have the following variants in your test app's `package.json`:

```json
Expand Down
Loading