From 36807c8f1fc9030a9568a3efed5afd8fd2e07789 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Wed, 31 Dec 2025 21:26:35 +0000 Subject: [PATCH 1/2] Add documentation for E2E testing setup and troubleshooting Co-authored-by: burak.kaya --- .cursor/rules/sdk_development.mdc | 59 ++++++++++++++++++++ CLAUDE.md | 59 ++++++++++++++++++++ CONTRIBUTING.md | 83 +++++++++++++++++++++++++++ dev-packages/e2e-tests/README.md | 93 +++++++++++++++++++++++++++++++ 4 files changed, 294 insertions(+) diff --git a/.cursor/rules/sdk_development.mdc b/.cursor/rules/sdk_development.mdc index 088c94f47a23..8a019b19e4b2 100644 --- a/.cursor/rules/sdk_development.mdc +++ b/.cursor/rules/sdk_development.mdc @@ -121,6 +121,65 @@ Each package typically contains: - Integration tests use Playwright extensively - Never change the volta, yarn, or package manager setup in general unless explicitly asked for +### E2E Testing + +E2E tests are located in `dev-packages/e2e-tests/` and verify SDK behavior in real-world framework scenarios. + +#### How Verdaccio Registry Works + +E2E tests use [Verdaccio](https://verdaccio.org/), a lightweight npm registry running in Docker. Before tests run: + +1. SDK packages are built and packed into tarballs (`yarn build && yarn build:tarball`) +2. Tarballs are published to Verdaccio at `http://127.0.0.1:4873` +3. Test applications install packages from Verdaccio instead of public npm + +#### Critical `.npmrc` Requirement + +**Every E2E test application MUST have an `.npmrc` file** with: + +``` +@sentry:registry=http://127.0.0.1:4873 +@sentry-internal:registry=http://127.0.0.1:4873 +``` + +Without this file, pnpm will install packages from the public npm registry, causing tests to use published versions instead of your local changes. This is a common cause of "tests pass in CI but fail locally" or vice versa. + +#### Running a Single E2E Test + +```bash +# Build packages first +yarn build && yarn build:tarball + +# Run a specific test app +cd dev-packages/e2e-tests +yarn test:run + +# Run with a specific variant (e.g., Next.js 15) +yarn test:run --variant +``` + +#### Common Pitfalls and Debugging + +1. **Missing `.npmrc`**: Most common issue. Always verify the test app has the correct `.npmrc` file. + +2. **Stale tarballs**: After SDK changes, must re-run `yarn build:tarball`. + +3. **Bundler environment variable handling**: + - Webpack's `DefinePlugin` doesn't replace values in `node_modules` + - Vite's `define` doesn't replace values in `node_modules` either + - For Vite apps needing env vars in dependencies, use `@rollup/plugin-replace` + - Next.js injects `process.env` via webpack/turbopack automatically + +4. **`import.meta.env` behavior**: + - Vite replaces `import.meta.env.VITE_*` at build time + - Other bundlers (webpack, turbopack) don't have `import.meta.env` + - SDK code must use try-catch when accessing `import.meta.env` + +5. **Debugging tips**: + - Check browser console logs for SDK initialization errors + - Use `debug: true` in Sentry config + - Verify installed package version: check `node_modules/@sentry/*/package.json` + ### Notes for Background Tasks - Make sure to use [volta](https://volta.sh/) for development. Volta is used to manage the node, yarn and pnpm version used. diff --git a/CLAUDE.md b/CLAUDE.md index e515c171303e..dd266db10f3c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -120,6 +120,65 @@ Each package typically contains: - Integration tests use Playwright extensively - Never change the volta, yarn, or package manager setup in general unless explicitly asked for +### E2E Testing + +E2E tests are located in `dev-packages/e2e-tests/` and verify SDK behavior in real-world framework scenarios. + +#### How Verdaccio Registry Works + +E2E tests use [Verdaccio](https://verdaccio.org/), a lightweight npm registry running in Docker. Before tests run: + +1. SDK packages are built and packed into tarballs (`yarn build && yarn build:tarball`) +2. Tarballs are published to Verdaccio at `http://127.0.0.1:4873` +3. Test applications install packages from Verdaccio instead of public npm + +#### Critical `.npmrc` Requirement + +**Every E2E test application MUST have an `.npmrc` file** with: + +``` +@sentry:registry=http://127.0.0.1:4873 +@sentry-internal:registry=http://127.0.0.1:4873 +``` + +Without this file, pnpm will install packages from the public npm registry, causing tests to use published versions instead of your local changes. This is a common cause of "tests pass in CI but fail locally" or vice versa. + +#### Running a Single E2E Test + +```bash +# Build packages first +yarn build && yarn build:tarball + +# Run a specific test app +cd dev-packages/e2e-tests +yarn test:run + +# Run with a specific variant (e.g., Next.js 15) +yarn test:run --variant +``` + +#### Common Pitfalls and Debugging + +1. **Missing `.npmrc`**: Most common issue. Always verify the test app has the correct `.npmrc` file. + +2. **Stale tarballs**: After SDK changes, must re-run `yarn build:tarball`. + +3. **Bundler environment variable handling**: + - Webpack's `DefinePlugin` doesn't replace values in `node_modules` + - Vite's `define` doesn't replace values in `node_modules` either + - For Vite apps needing env vars in dependencies, use `@rollup/plugin-replace` + - Next.js injects `process.env` via webpack/turbopack automatically + +4. **`import.meta.env` behavior**: + - Vite replaces `import.meta.env.VITE_*` at build time + - Other bundlers (webpack, turbopack) don't have `import.meta.env` + - SDK code must use try-catch when accessing `import.meta.env` + +5. **Debugging tips**: + - Check browser console logs for SDK initialization errors + - Use `debug: true` in Sentry config + - Verify installed package version: check `node_modules/@sentry/*/package.json` + ### Notes for Background Tasks - Make sure to use [volta](https://volta.sh/) for development. Volta is used to manage the node, yarn and pnpm version used. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 8d486d6718c1..70ebd45da74f 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -73,6 +73,89 @@ the tests in each location. Check out the `scripts` entry of the corresponding ` Note: you must run `yarn build` before `yarn test` will work. +## Running E2E Tests Locally + +E2E tests verify SDK behavior in real-world framework scenarios using a local npm registry (Verdaccio). + +### Prerequisites + +1. **Docker**: Required to run the Verdaccio registry container +2. **Volta with pnpm support**: Enable pnpm in Volta by setting `VOLTA_FEATURE_PNPM=1` in your environment. See [Volta pnpm docs](https://docs.volta.sh/advanced/pnpm). + +### Step-by-Step Instructions + +1. **Build the SDK packages and create tarballs:** + + ```bash + yarn build + yarn build:tarball + ``` + + Note: You must re-run `yarn build:tarball` after any changes to packages. + +2. **Set up environment (optional):** + + ```bash + cd dev-packages/e2e-tests + cp .env.example .env + # Fill in Sentry project auth info if running tests that send data to Sentry + ``` + +3. **Run all E2E tests:** + + ```bash + yarn test:e2e + ``` + +4. **Or run a specific test application:** + + ```bash + yarn test:run + # Example: yarn test:run nextjs-app-dir + ``` + +5. **Run with a specific variant:** + ```bash + yarn test:run --variant + # Example: yarn test:run nextjs-pages-dir --variant 15 + ``` + +### Common Issues and Troubleshooting + +#### Packages install from public npm instead of Verdaccio + +Every E2E test application **must** have an `.npmrc` file with: + +``` +@sentry:registry=http://127.0.0.1:4873 +@sentry-internal:registry=http://127.0.0.1:4873 +``` + +Without this, pnpm will fetch packages from the public npm registry instead of the local Verdaccio instance, causing tests to use outdated/published versions instead of your local changes. + +#### Tests fail after making SDK changes + +Make sure to rebuild tarballs: + +```bash +yarn build +yarn build:tarball +``` + +#### Docker-related issues + +- Ensure Docker daemon is running +- Check that port 4873 is not in use by another process +- Try stopping and removing existing Verdaccio containers + +#### Debugging test failures + +1. Check browser console logs for SDK initialization errors +2. Enable debug mode in the test app's Sentry config: `debug: true` +3. Verify packages are installed from Verdaccio by checking the version in `node_modules/@sentry/*/package.json` + +For more details, see [dev-packages/e2e-tests/README.md](dev-packages/e2e-tests/README.md). + ## Debug Build Flags Throughout the codebase, you will find a `__DEBUG_BUILD__` constant. This flag serves two purposes: diff --git a/dev-packages/e2e-tests/README.md b/dev-packages/e2e-tests/README.md index 133b53268d52..5d8d196f6aa6 100644 --- a/dev-packages/e2e-tests/README.md +++ b/dev-packages/e2e-tests/README.md @@ -88,6 +88,99 @@ EOF Make sure to add a `test:build` and `test:assert` command to the new app's `package.json` file. +### Critical: The `.npmrc` File + +**Every E2E test application MUST have an `.npmrc` file.** This file tells pnpm to fetch `@sentry/*` and `@sentry-internal/*` packages from the local Verdaccio registry instead of the public npm registry. + +The `.npmrc` file must contain: + +``` +@sentry:registry=http://127.0.0.1:4873 +@sentry-internal:registry=http://127.0.0.1:4873 +``` + +**Why is this critical?** + +Without this file: + +- pnpm will install packages from the public npm registry +- Your local SDK changes will NOT be tested +- Tests may pass or fail based on the published version, not your changes +- This is one of the most common causes of confusing test failures + +**How to verify packages are installed from Verdaccio:** + +Check the version in `node_modules/@sentry/*/package.json`. If it shows a version like `0.0.0-pr.12345` or matches your local build, Verdaccio is working correctly. If it shows a released version (e.g., `8.0.0`), the `.npmrc` is missing or incorrect. + +## Troubleshooting + +### Common Issues + +#### Tests fail with "Cannot find module '@sentry/...'" or use wrong package version + +1. Verify the test application has an `.npmrc` file (see above) +2. Rebuild tarballs: `yarn build && yarn build:tarball` +3. Delete `node_modules` in the test application and re-run the test + +#### Docker/Verdaccio issues + +- Ensure Docker daemon is running +- Check that port 4873 is not already in use: `lsof -i :4873` +- Stop any existing Verdaccio containers: `docker ps` and `docker stop ` +- Check Verdaccio logs for errors + +#### Tests pass locally but fail in CI (or vice versa) + +- Most likely cause: missing `.npmrc` file +- Verify all `@sentry/*` dependencies use `latest || *` version specifier +- Check if the test relies on environment-specific behavior + +### Debugging Tips + +1. **Enable Sentry debug mode**: Add `debug: true` to the Sentry init config to see detailed SDK logs +2. **Check browser console**: Look for SDK initialization errors or warnings +3. **Inspect network requests**: Verify events are being sent to the expected endpoint +4. **Check installed versions**: `cat node_modules/@sentry/browser/package.json | grep version` + +## Bundler-Specific Behavior + +Different bundlers handle environment variables and code replacement differently. This is important when writing tests or SDK code that relies on build-time constants. + +### Webpack + +- `DefinePlugin` replaces variables in your application code +- **Does NOT replace values inside `node_modules`** +- Environment variables must be explicitly defined + +### Vite + +- `define` option replaces variables in your application code +- **Does NOT replace values inside `node_modules`** +- `import.meta.env.VITE_*` variables are replaced at build time +- For replacing values in dependencies, use `@rollup/plugin-replace` + +### Next.js + +- Automatically injects `process.env` via webpack/turbopack +- Handles environment variables more seamlessly than raw webpack/Vite +- Server and client bundles may have different environment variable access + +### `import.meta.env` Considerations + +- Only available in Vite and ES modules +- Webpack and Turbopack do not have `import.meta.env` +- SDK code accessing `import.meta.env` must use try-catch to handle environments where it doesn't exist + +```typescript +// Safe pattern for SDK code +let envValue: string | undefined; +try { + envValue = import.meta.env.VITE_SOME_VAR; +} catch { + // import.meta.env not available in this bundler +} +``` + Test apps in the folder `test-applications` will be automatically picked up by CI in the job `job_e2e_tests` (in `.github/workflows/build.yml`). The test matrix for CI is generated in `dev-packages/e2e-tests/lib/getTestMatrix.ts`. From a077eeaab6ce535b4585a0d81cc668b5e1290560 Mon Sep 17 00:00:00 2001 From: Burak Yigit Kaya Date: Tue, 6 Jan 2026 13:56:29 +0000 Subject: [PATCH 2/2] review comments --- .cursor/rules/sdk_development.mdc | 19 ++++--------------- CLAUDE.md | 19 ++++--------------- dev-packages/e2e-tests/README.md | 24 +++--------------------- 3 files changed, 11 insertions(+), 51 deletions(-) diff --git a/.cursor/rules/sdk_development.mdc b/.cursor/rules/sdk_development.mdc index 8a019b19e4b2..c997b65f5482 100644 --- a/.cursor/rules/sdk_development.mdc +++ b/.cursor/rules/sdk_development.mdc @@ -133,16 +133,16 @@ E2E tests use [Verdaccio](https://verdaccio.org/), a lightweight npm registry ru 2. Tarballs are published to Verdaccio at `http://127.0.0.1:4873` 3. Test applications install packages from Verdaccio instead of public npm -#### Critical `.npmrc` Requirement +#### The `.npmrc` Requirement -**Every E2E test application MUST have an `.npmrc` file** with: +Every E2E test application needs an `.npmrc` file with: ``` @sentry:registry=http://127.0.0.1:4873 @sentry-internal:registry=http://127.0.0.1:4873 ``` -Without this file, pnpm will install packages from the public npm registry, causing tests to use published versions instead of your local changes. This is a common cause of "tests pass in CI but fail locally" or vice versa. +Without this file, pnpm installs from the public npm registry instead of Verdaccio, so your local changes won't be tested. This is a common cause of "tests pass in CI but fail locally" or vice versa. #### Running a Single E2E Test @@ -164,18 +164,7 @@ yarn test:run --variant 2. **Stale tarballs**: After SDK changes, must re-run `yarn build:tarball`. -3. **Bundler environment variable handling**: - - Webpack's `DefinePlugin` doesn't replace values in `node_modules` - - Vite's `define` doesn't replace values in `node_modules` either - - For Vite apps needing env vars in dependencies, use `@rollup/plugin-replace` - - Next.js injects `process.env` via webpack/turbopack automatically - -4. **`import.meta.env` behavior**: - - Vite replaces `import.meta.env.VITE_*` at build time - - Other bundlers (webpack, turbopack) don't have `import.meta.env` - - SDK code must use try-catch when accessing `import.meta.env` - -5. **Debugging tips**: +3. **Debugging tips**: - Check browser console logs for SDK initialization errors - Use `debug: true` in Sentry config - Verify installed package version: check `node_modules/@sentry/*/package.json` diff --git a/CLAUDE.md b/CLAUDE.md index dd266db10f3c..cae60376d964 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -132,16 +132,16 @@ E2E tests use [Verdaccio](https://verdaccio.org/), a lightweight npm registry ru 2. Tarballs are published to Verdaccio at `http://127.0.0.1:4873` 3. Test applications install packages from Verdaccio instead of public npm -#### Critical `.npmrc` Requirement +#### The `.npmrc` Requirement -**Every E2E test application MUST have an `.npmrc` file** with: +Every E2E test application needs an `.npmrc` file with: ``` @sentry:registry=http://127.0.0.1:4873 @sentry-internal:registry=http://127.0.0.1:4873 ``` -Without this file, pnpm will install packages from the public npm registry, causing tests to use published versions instead of your local changes. This is a common cause of "tests pass in CI but fail locally" or vice versa. +Without this file, pnpm installs from the public npm registry instead of Verdaccio, so your local changes won't be tested. This is a common cause of "tests pass in CI but fail locally" or vice versa. #### Running a Single E2E Test @@ -163,18 +163,7 @@ yarn test:run --variant 2. **Stale tarballs**: After SDK changes, must re-run `yarn build:tarball`. -3. **Bundler environment variable handling**: - - Webpack's `DefinePlugin` doesn't replace values in `node_modules` - - Vite's `define` doesn't replace values in `node_modules` either - - For Vite apps needing env vars in dependencies, use `@rollup/plugin-replace` - - Next.js injects `process.env` via webpack/turbopack automatically - -4. **`import.meta.env` behavior**: - - Vite replaces `import.meta.env.VITE_*` at build time - - Other bundlers (webpack, turbopack) don't have `import.meta.env` - - SDK code must use try-catch when accessing `import.meta.env` - -5. **Debugging tips**: +3. **Debugging tips**: - Check browser console logs for SDK initialization errors - Use `debug: true` in Sentry config - Verify installed package version: check `node_modules/@sentry/*/package.json` diff --git a/dev-packages/e2e-tests/README.md b/dev-packages/e2e-tests/README.md index 5d8d196f6aa6..ffe06dd91aaf 100644 --- a/dev-packages/e2e-tests/README.md +++ b/dev-packages/e2e-tests/README.md @@ -88,29 +88,11 @@ EOF Make sure to add a `test:build` and `test:assert` command to the new app's `package.json` file. -### Critical: The `.npmrc` File +### The `.npmrc` File -**Every E2E test application MUST have an `.npmrc` file.** This file tells pnpm to fetch `@sentry/*` and `@sentry-internal/*` packages from the local Verdaccio registry instead of the public npm registry. +Every test application needs an `.npmrc` file (as shown above) to tell pnpm to fetch `@sentry/*` and `@sentry-internal/*` packages from the local Verdaccio registry. Without it, pnpm will install from the public npm registry and your local changes won't be tested - this is one of the most common causes of confusing test failures. -The `.npmrc` file must contain: - -``` -@sentry:registry=http://127.0.0.1:4873 -@sentry-internal:registry=http://127.0.0.1:4873 -``` - -**Why is this critical?** - -Without this file: - -- pnpm will install packages from the public npm registry -- Your local SDK changes will NOT be tested -- Tests may pass or fail based on the published version, not your changes -- This is one of the most common causes of confusing test failures - -**How to verify packages are installed from Verdaccio:** - -Check the version in `node_modules/@sentry/*/package.json`. If it shows a version like `0.0.0-pr.12345` or matches your local build, Verdaccio is working correctly. If it shows a released version (e.g., `8.0.0`), the `.npmrc` is missing or incorrect. +To verify packages are being installed from Verdaccio, check the version in `node_modules/@sentry/*/package.json`. If it shows something like `0.0.0-pr.12345`, Verdaccio is working. If it shows a released version (e.g., `8.0.0`), the `.npmrc` is missing or incorrect. ## Troubleshooting