Skip to content

Commit 36807c8

Browse files
cursoragentBYK
cursoragent
and
BYK
committed
Add documentation for E2E testing setup and troubleshooting
Co-authored-by: burak.kaya <burak.kaya@sentry.io>
1 parent 4a5ba93 commit 36807c8

File tree

4 files changed

+294
-0
lines changed

4 files changed

+294
-0
lines changed

‎.cursor/rules/sdk_development.mdc‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -121,6 +121,65 @@ Each package typically contains:
121121
- Integration tests use Playwright extensively
122122
- Never change the volta, yarn, or package manager setup in general unless explicitly asked for
123123

124+
### E2E Testing
125+
126+
E2E tests are located in `dev-packages/e2e-tests/` and verify SDK behavior in real-world framework scenarios.
127+
128+
#### How Verdaccio Registry Works
129+
130+
E2E tests use [Verdaccio](https://verdaccio.org/), a lightweight npm registry running in Docker. Before tests run:
131+
132+
1. SDK packages are built and packed into tarballs (`yarn build && yarn build:tarball`)
133+
2. Tarballs are published to Verdaccio at `http://127.0.0.1:4873`
134+
3. Test applications install packages from Verdaccio instead of public npm
135+
136+
#### Critical `.npmrc` Requirement
137+
138+
**Every E2E test application MUST have an `.npmrc` file** with:
139+
140+
```
141+
@sentry:registry=http://127.0.0.1:4873
142+
@sentry-internal:registry=http://127.0.0.1:4873
143+
```
144+
145+
Without this file, pnpm will install packages from the public npm registry, causing tests to use published versions instead of your local changes. This is a common cause of "tests pass in CI but fail locally" or vice versa.
146+
147+
#### Running a Single E2E Test
148+
149+
```bash
150+
# Build packages first
151+
yarn build && yarn build:tarball
152+
153+
# Run a specific test app
154+
cd dev-packages/e2e-tests
155+
yarn test:run <app-name>
156+
157+
# Run with a specific variant (e.g., Next.js 15)
158+
yarn test:run <app-name> --variant <variant-name>
159+
```
160+
161+
#### Common Pitfalls and Debugging
162+
163+
1. **Missing `.npmrc`**: Most common issue. Always verify the test app has the correct `.npmrc` file.
164+
165+
2. **Stale tarballs**: After SDK changes, must re-run `yarn build:tarball`.
166+
167+
3. **Bundler environment variable handling**:
168+
- Webpack's `DefinePlugin` doesn't replace values in `node_modules`
169+
- Vite's `define` doesn't replace values in `node_modules` either
170+
- For Vite apps needing env vars in dependencies, use `@rollup/plugin-replace`
171+
- Next.js injects `process.env` via webpack/turbopack automatically
172+
173+
4. **`import.meta.env` behavior**:
174+
- Vite replaces `import.meta.env.VITE_*` at build time
175+
- Other bundlers (webpack, turbopack) don't have `import.meta.env`
176+
- SDK code must use try-catch when accessing `import.meta.env`
177+
178+
5. **Debugging tips**:
179+
- Check browser console logs for SDK initialization errors
180+
- Use `debug: true` in Sentry config
181+
- Verify installed package version: check `node_modules/@sentry/*/package.json`
182+
124183
### Notes for Background Tasks
125184

126185
- Make sure to use [volta](https://volta.sh/) for development. Volta is used to manage the node, yarn and pnpm version used.

‎CLAUDE.md‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,65 @@ Each package typically contains:
120120
- Integration tests use Playwright extensively
121121
- Never change the volta, yarn, or package manager setup in general unless explicitly asked for
122122

123+
### E2E Testing
124+
125+
E2E tests are located in `dev-packages/e2e-tests/` and verify SDK behavior in real-world framework scenarios.
126+
127+
#### How Verdaccio Registry Works
128+
129+
E2E tests use [Verdaccio](https://verdaccio.org/), a lightweight npm registry running in Docker. Before tests run:
130+
131+
1. SDK packages are built and packed into tarballs (`yarn build && yarn build:tarball`)
132+
2. Tarballs are published to Verdaccio at `http://127.0.0.1:4873`
133+
3. Test applications install packages from Verdaccio instead of public npm
134+
135+
#### Critical `.npmrc` Requirement
136+
137+
**Every E2E test application MUST have an `.npmrc` file** with:
138+
139+
```
140+
@sentry:registry=http://127.0.0.1:4873
141+
@sentry-internal:registry=http://127.0.0.1:4873
142+
```
143+
144+
Without this file, pnpm will install packages from the public npm registry, causing tests to use published versions instead of your local changes. This is a common cause of "tests pass in CI but fail locally" or vice versa.
145+
146+
#### Running a Single E2E Test
147+
148+
```bash
149+
# Build packages first
150+
yarn build && yarn build:tarball
151+
152+
# Run a specific test app
153+
cd dev-packages/e2e-tests
154+
yarn test:run <app-name>
155+
156+
# Run with a specific variant (e.g., Next.js 15)
157+
yarn test:run <app-name> --variant <variant-name>
158+
```
159+
160+
#### Common Pitfalls and Debugging
161+
162+
1. **Missing `.npmrc`**: Most common issue. Always verify the test app has the correct `.npmrc` file.
163+
164+
2. **Stale tarballs**: After SDK changes, must re-run `yarn build:tarball`.
165+
166+
3. **Bundler environment variable handling**:
167+
- Webpack's `DefinePlugin` doesn't replace values in `node_modules`
168+
- Vite's `define` doesn't replace values in `node_modules` either
169+
- For Vite apps needing env vars in dependencies, use `@rollup/plugin-replace`
170+
- Next.js injects `process.env` via webpack/turbopack automatically
171+
172+
4. **`import.meta.env` behavior**:
173+
- Vite replaces `import.meta.env.VITE_*` at build time
174+
- Other bundlers (webpack, turbopack) don't have `import.meta.env`
175+
- SDK code must use try-catch when accessing `import.meta.env`
176+
177+
5. **Debugging tips**:
178+
- Check browser console logs for SDK initialization errors
179+
- Use `debug: true` in Sentry config
180+
- Verify installed package version: check `node_modules/@sentry/*/package.json`
181+
123182
### Notes for Background Tasks
124183

125184
- Make sure to use [volta](https://volta.sh/) for development. Volta is used to manage the node, yarn and pnpm version used.

‎CONTRIBUTING.md‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,89 @@ the tests in each location. Check out the `scripts` entry of the corresponding `
7373

7474
Note: you must run `yarn build` before `yarn test` will work.
7575

76+
## Running E2E Tests Locally
77+
78+
E2E tests verify SDK behavior in real-world framework scenarios using a local npm registry (Verdaccio).
79+
80+
### Prerequisites
81+
82+
1. **Docker**: Required to run the Verdaccio registry container
83+
2. **Volta with pnpm support**: Enable pnpm in Volta by setting `VOLTA_FEATURE_PNPM=1` in your environment. See [Volta pnpm docs](https://docs.volta.sh/advanced/pnpm).
84+
85+
### Step-by-Step Instructions
86+
87+
1. **Build the SDK packages and create tarballs:**
88+
89+
```bash
90+
yarn build
91+
yarn build:tarball
92+
```
93+
94+
Note: You must re-run `yarn build:tarball` after any changes to packages.
95+
96+
2. **Set up environment (optional):**
97+
98+
```bash
99+
cd dev-packages/e2e-tests
100+
cp .env.example .env
101+
# Fill in Sentry project auth info if running tests that send data to Sentry
102+
```
103+
104+
3. **Run all E2E tests:**
105+
106+
```bash
107+
yarn test:e2e
108+
```
109+
110+
4. **Or run a specific test application:**
111+
112+
```bash
113+
yarn test:run <app-name>
114+
# Example: yarn test:run nextjs-app-dir
115+
```
116+
117+
5. **Run with a specific variant:**
118+
```bash
119+
yarn test:run <app-name> --variant <variant-name>
120+
# Example: yarn test:run nextjs-pages-dir --variant 15
121+
```
122+
123+
### Common Issues and Troubleshooting
124+
125+
#### Packages install from public npm instead of Verdaccio
126+
127+
Every E2E test application **must** have an `.npmrc` file with:
128+
129+
```
130+
@sentry:registry=http://127.0.0.1:4873
131+
@sentry-internal:registry=http://127.0.0.1:4873
132+
```
133+
134+
Without this, pnpm will fetch packages from the public npm registry instead of the local Verdaccio instance, causing tests to use outdated/published versions instead of your local changes.
135+
136+
#### Tests fail after making SDK changes
137+
138+
Make sure to rebuild tarballs:
139+
140+
```bash
141+
yarn build
142+
yarn build:tarball
143+
```
144+
145+
#### Docker-related issues
146+
147+
- Ensure Docker daemon is running
148+
- Check that port 4873 is not in use by another process
149+
- Try stopping and removing existing Verdaccio containers
150+
151+
#### Debugging test failures
152+
153+
1. Check browser console logs for SDK initialization errors
154+
2. Enable debug mode in the test app's Sentry config: `debug: true`
155+
3. Verify packages are installed from Verdaccio by checking the version in `node_modules/@sentry/*/package.json`
156+
157+
For more details, see [dev-packages/e2e-tests/README.md](dev-packages/e2e-tests/README.md).
158+
76159
## Debug Build Flags
77160

78161
Throughout the codebase, you will find a `__DEBUG_BUILD__` constant. This flag serves two purposes:

‎dev-packages/e2e-tests/README.md‎

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,99 @@ EOF
8888

8989
Make sure to add a `test:build` and `test:assert` command to the new app's `package.json` file.
9090

91+
### Critical: The `.npmrc` File
92+
93+
**Every E2E test application MUST have an `.npmrc` file.** This file tells pnpm to fetch `@sentry/*` and `@sentry-internal/*` packages from the local Verdaccio registry instead of the public npm registry.
94+
95+
The `.npmrc` file must contain:
96+
97+
```
98+
@sentry:registry=http://127.0.0.1:4873
99+
@sentry-internal:registry=http://127.0.0.1:4873
100+
```
101+
102+
**Why is this critical?**
103+
104+
Without this file:
105+
106+
- pnpm will install packages from the public npm registry
107+
- Your local SDK changes will NOT be tested
108+
- Tests may pass or fail based on the published version, not your changes
109+
- This is one of the most common causes of confusing test failures
110+
111+
**How to verify packages are installed from Verdaccio:**
112+
113+
Check the version in `node_modules/@sentry/*/package.json`. If it shows a version like `0.0.0-pr.12345` or matches your local build, Verdaccio is working correctly. If it shows a released version (e.g., `8.0.0`), the `.npmrc` is missing or incorrect.
114+
115+
## Troubleshooting
116+
117+
### Common Issues
118+
119+
#### Tests fail with "Cannot find module '@sentry/...'" or use wrong package version
120+
121+
1. Verify the test application has an `.npmrc` file (see above)
122+
2. Rebuild tarballs: `yarn build && yarn build:tarball`
123+
3. Delete `node_modules` in the test application and re-run the test
124+
125+
#### Docker/Verdaccio issues
126+
127+
- Ensure Docker daemon is running
128+
- Check that port 4873 is not already in use: `lsof -i :4873`
129+
- Stop any existing Verdaccio containers: `docker ps` and `docker stop <container-id>`
130+
- Check Verdaccio logs for errors
131+
132+
#### Tests pass locally but fail in CI (or vice versa)
133+
134+
- Most likely cause: missing `.npmrc` file
135+
- Verify all `@sentry/*` dependencies use `latest || *` version specifier
136+
- Check if the test relies on environment-specific behavior
137+
138+
### Debugging Tips
139+
140+
1. **Enable Sentry debug mode**: Add `debug: true` to the Sentry init config to see detailed SDK logs
141+
2. **Check browser console**: Look for SDK initialization errors or warnings
142+
3. **Inspect network requests**: Verify events are being sent to the expected endpoint
143+
4. **Check installed versions**: `cat node_modules/@sentry/browser/package.json | grep version`
144+
145+
## Bundler-Specific Behavior
146+
147+
Different bundlers handle environment variables and code replacement differently. This is important when writing tests or SDK code that relies on build-time constants.
148+
149+
### Webpack
150+
151+
- `DefinePlugin` replaces variables in your application code
152+
- **Does NOT replace values inside `node_modules`**
153+
- Environment variables must be explicitly defined
154+
155+
### Vite
156+
157+
- `define` option replaces variables in your application code
158+
- **Does NOT replace values inside `node_modules`**
159+
- `import.meta.env.VITE_*` variables are replaced at build time
160+
- For replacing values in dependencies, use `@rollup/plugin-replace`
161+
162+
### Next.js
163+
164+
- Automatically injects `process.env` via webpack/turbopack
165+
- Handles environment variables more seamlessly than raw webpack/Vite
166+
- Server and client bundles may have different environment variable access
167+
168+
### `import.meta.env` Considerations
169+
170+
- Only available in Vite and ES modules
171+
- Webpack and Turbopack do not have `import.meta.env`
172+
- SDK code accessing `import.meta.env` must use try-catch to handle environments where it doesn't exist
173+
174+
```typescript
175+
// Safe pattern for SDK code
176+
let envValue: string | undefined;
177+
try {
178+
envValue = import.meta.env.VITE_SOME_VAR;
179+
} catch {
180+
// import.meta.env not available in this bundler
181+
}
182+
```
183+
91184
Test apps in the folder `test-applications` will be automatically picked up by CI in the job `job_e2e_tests` (in `.github/workflows/build.yml`).
92185
The test matrix for CI is generated in `dev-packages/e2e-tests/lib/getTestMatrix.ts`.
93186

0 commit comments

Comments
 (0)