Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
156 changes: 39 additions & 117 deletions .claude/commands/ci-failures.md
Original file line number Diff line number Diff line change
@@ -1,91 +1,76 @@
# Check CI Failures

Analyze failing tests from PR CI runs with parallel subagent log analysis.
Analyze failing tests from PR CI runs.

## Usage

```
/ci-failures [pr-number]
/ci-failures
```

If no PR number provided, detect from current branch.
Automatically detects PR from current branch.

## Instructions

1. Get the PR number from argument or current branch:
1. Run the script to fetch CI failure data:

```bash
gh pr view --json number,headRefName --jq '"\(.number) \(.headRefName)"'
node scripts/ci-failures.js
```

2. **CRITICAL: Always fetch fresh run IDs** - never trust cached IDs from conversation summaries:
This fetches workflow runs, failed jobs, and logs, then generates markdown files.

2. Read the generated index file for a summary:

```bash
gh api "repos/vercel/next.js/actions/runs?branch={branch}&per_page=10" \
--jq '.workflow_runs[] | select(.name == "build-and-test") | "\(.id) attempts:\(.run_attempt) status:\(.status) conclusion:\(.conclusion)"'
# Read scripts/ci-failures/index.md
```

3. **Prioritize the MOST RECENT run, even if in-progress:**
- If the latest run is `in_progress` or `queued`, check it FIRST - it has the most relevant failures
- Individual jobs complete before the overall run - analyze them as they finish
- Only fall back to older completed runs if the current run has no completed jobs yet
The index shows all failed jobs with links to details.

4. Get all failed jobs from the run (works for in-progress runs too):
3. Spawn parallel haiku subagents to analyze the failing jobs (limit to 3-4 to avoid rate limits)
- **Agent prompt template** (copy-paste for each agent):

```bash
gh api "repos/vercel/next.js/actions/runs/{run_id}/jobs?per_page=100" \
--jq '.jobs[] | select(.conclusion == "failure") | "\(.id) \(.name)"'
```
Analyze CI results for these jobs: scripts/ci-failures/job-{id1}.md scripts/ci-failures/job-{id2}.md
For each failing test, extract:
1. TEST FILE: (full path, e.g., test/production/required-server-files-ssr-404/test/index.test.ts)
2. TEST NAME: (the specific test case name)1
3. JOB TYPE: (the kind of the job, e.g. turbopack production, webpack dev, rust check)
4. EXPECTED: (exact expected value from assertion)
5. RECEIVED: (exact received value from assertion)
6. CATEGORY: (assertion|timeout|routing|source-map|build|cli-output)
7. ROOT CAUSE: (one sentence hypothesis)
8. LOG FILE: (analysed log file that led to conclusion)
Return structured findings grouped by TEST FILE, not by job.

Also extract other failures that are not related to tests.
Identify if they are likely transient.

**Note:** For runs with >100 jobs, paginate:

```bash
gh api "repos/vercel/next.js/actions/runs/{run_id}/jobs?per_page=100&page=2"
```

5. Spawn parallel haiku subagents to analyze logs (limit to 3-4 to avoid rate limits):
- **CRITICAL: Use the API endpoint for logs, NOT `gh run view`**
- `gh run view --job --log` FAILS when run is in-progress
- **Do NOT group by job name** (e.g., "test dev", "turbopack") - group by failure pattern instead
- Agent prompt should extract structured data using:
```bash
# Extract assertion failures with context:
gh api "repos/vercel/next.js/actions/jobs/{job_id}/logs" 2>&1 | \
grep -B3 -A10 "expect.*\(toBe\|toContain\|toEqual\|toStartWith\|toMatch\)" | head -100
# Also check for test file paths:
gh api "repos/vercel/next.js/actions/jobs/{job_id}/logs" 2>&1 | \
grep -E "^\s+at Object\.|FAIL\s+test/" | head -20
```
- **Agent prompt template** (copy-paste for each agent):
```
Analyze CI logs for these jobs: {job_ids}
For each failing test, extract:
1. TEST FILE: (full path, e.g., test/production/required-server-files-ssr-404/test/index.test.ts)
2. TEST NAME: (the specific test case name)
3. EXPECTED: (exact expected value from assertion)
4. RECEIVED: (exact received value from assertion)
5. CATEGORY: (assertion|timeout|routing|source-map|build|cli-output)
6. ROOT CAUSE: (one sentence hypothesis)
Return structured findings grouped by TEST FILE, not by job.
```

6. **Deduplicate by test file** before summarizing:
4. **Deduplicate by test file** before summarizing:
- Group all failures by TEST FILE path, not by CI job name
- If multiple jobs fail the same test file, count them but report once
- Identify systemic issues (same test failing across many jobs)

7. Create summary table **grouped by test file**:
| Test File | Issue (Expected vs Received) | Jobs | Priority |
|-----------|------------------------------|------|----------|
| `test/production/required-server-files-ssr-404/...` | `"second"` vs `"[slug]"` (routing) | 3 | HIGH |
| `test/integration/server-side-dev-errors/...` | source map paths wrong | 5 | HIGH |
| `test/e2e/app-dir/disable-logging-route/...` | "Compiling" appearing when disabled | 2 | MEDIUM |
5. Analyze failures and create a summary **grouped by test file**:

8. Recommend fixes:
| Test File | Type. | Issue (Expected vs Received) | Jobs | Priority |
| --------------------------------------------------- | -------------- | ----------------------------------- | ---- | -------- |
| `test/production/required-server-files-ssr-404/...` | Turbopack prod | `"second"` vs `"[slug]"` (routing) | 3 | HIGH |
| `test/integration/server-side-dev-errors/...` | webpack dev | source map paths wrong | 5 | HIGH |
| `test/e2e/app-dir/disable-logging-route/...` | prod | "Compiling" appearing when disabled | 2 | MEDIUM |
| N/A | rust check | Formatting incorrect | 2 | MEDIUM |

6. Recommend fixes:
- **HIGH priority**: Show specific expected vs actual values, include test file path
- **MEDIUM priority**: Identify root cause pattern
- **LOW priority**: Mark as likely flaky/transient

- Do not try to fix these failures.
- If failures would require complex analysis and there are multiple problems, only do some basic analysis and point out that further investigation is needed and could be performant when requested.

## Failure Categories

- **Infrastructure/Transient**: Network errors, 503s, timeouts unrelated to code
Expand All @@ -96,66 +81,3 @@ If no PR number provided, detect from current branch.
- **Routing/SSR**: Dynamic params not resolved, wrong status codes, JSON parse errors
- **Source Maps**: `webpack-internal://` paths, wrong line numbers, missing code frames
- **CLI Output**: Missing warnings, wrong log order, "Ready" printed before errors

## Failure Extraction Patterns

Use these grep patterns to identify specific failure types:

```bash
# Assertion failures (most common)
grep -B3 -A10 "expect.*\(toBe\|toContain\|toEqual\|toStartWith\)" | head -100

# Routing issues (dynamic params, status codes)
grep -E "Expected.*Received|\[slug\]|x-matched-path|Expected: [0-9]+" | head -50

# Source map issues
grep -E "webpack-internal://|at .* \(webpack" | head -30

# CLI output issues (missing warnings)
grep -E "Ready in|deprecated|Both middleware|Compiling" | head -30

# Timeout issues
grep -E "TIMEOUT|TimeoutError|exceeded|Exceeded timeout" | head -20

# Test file paths (to identify which test is failing)
grep -E "FAIL test/|at Object\.<anonymous> \(" | head -20
```

## Common Gotchas

### In-Progress Runs

- `gh run view {run_id} --job {job_id} --log` **FAILS** when run is in-progress
- `gh api "repos/.../actions/jobs/{job_id}/logs"` **WORKS** for any completed job
- Always use the API endpoint for reliability

### Pagination

- GitHub API paginates at 100 jobs per page
- Next.js CI has ~120+ jobs - always check page 2:
```bash
gh api ".../jobs?per_page=100&page=1" --jq '[.jobs[] | select(.conclusion == "failure")] | length'
gh api ".../jobs?per_page=100&page=2" --jq '[.jobs[] | select(.conclusion == "failure")] | length'
```

### Multiple Attempts

- CI runs can have multiple attempts (retries)
- Check attempt count: `.run_attempt` field
- Query specific attempt: `.../runs/{id}/attempts/{n}/jobs`
- 404 on attempt endpoint means that attempt doesn't exist

## Quick Reference

```bash
# Get failed jobs (works for in-progress runs)
gh api "repos/vercel/next.js/actions/runs/{run_id}/jobs?per_page=100" \
--jq '.jobs[] | select(.conclusion == "failure") | "\(.id) \(.name)"'

# Get logs for a specific job (works for in-progress runs)
gh api "repos/vercel/next.js/actions/jobs/{job_id}/logs" 2>&1 | head -500

# Search logs for errors
gh api "repos/vercel/next.js/actions/jobs/{job_id}/logs" 2>&1 | \
grep -E "FAIL|Error|error:|✕|Expected|Received" | head -50
```
12 changes: 8 additions & 4 deletions crates/next-core/src/next_client/context.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,12 @@ use turbopack::module_options::{
side_effect_free_packages_glob,
};
use turbopack_browser::{
BrowserChunkingContext, ChunkSuffix, ContentHashing, CurrentChunkMethod,
BrowserChunkingContext, ContentHashing, CurrentChunkMethod,
react_refresh::assert_can_resolve_react_refresh,
};
use turbopack_core::{
chunk::{
ChunkingConfig, ChunkingContext, MangleType, MinifyType, SourceMapSourceType,
AssetSuffix, ChunkingConfig, ChunkingContext, MangleType, MinifyType, SourceMapSourceType,
SourceMapsType, UnusedReferences, chunk_id_strategy::ModuleIdStrategy,
},
compile_time_info::{CompileTimeDefines, CompileTimeInfo, FreeVarReference, FreeVarReferences},
Expand All @@ -26,7 +26,9 @@ use turbopack_core::{
resolve::{parse::Request, pattern::Pattern},
};
use turbopack_css::chunk::CssChunkType;
use turbopack_ecmascript::{AnalyzeMode, TypeofWindow, chunk::EcmascriptChunkType};
use turbopack_ecmascript::{
AnalyzeMode, TypeofWindow, chunk::EcmascriptChunkType, references::esm::UrlRewriteBehavior,
};
use turbopack_node::{
execution_context::ExecutionContext,
transforms::postcss::{PostCssConfigLocation, PostCssTransformOptions},
Expand Down Expand Up @@ -323,6 +325,7 @@ pub async fn get_client_module_options_context(
let source_maps = *next_config.client_source_maps(mode).await?;
let module_options_context = ModuleOptionsContext {
ecmascript: EcmascriptOptionsContext {
esm_url_rewrite_behavior: Some(UrlRewriteBehavior::Relative),
enable_typeof_window_inlining: Some(TypeofWindow::Object),
source_maps,
infer_module_side_effects: *next_config.turbopack_infer_module_side_effects().await?,
Expand All @@ -333,6 +336,7 @@ pub async fn get_client_module_options_context(
module_css_condition: Some(module_styles_rule_condition()),
..Default::default()
},
static_url_tag: Some(rcstr!("client")),
environment: Some(env),
execution_context: Some(execution_context),
tree_shaking_mode: tree_shaking_mode_for_user_code,
Expand Down Expand Up @@ -468,7 +472,7 @@ pub async fn get_client_chunking_context(
next_mode.runtime_type(),
)
.chunk_base_path(Some(asset_prefix.clone()))
.chunk_suffix(ChunkSuffix::FromScriptSrc.resolved_cell())
.asset_suffix(AssetSuffix::Inferred.resolved_cell())
.minify_type(if *minify.await? {
MinifyType::Minify {
mangle: (!*no_mangling.await?).then_some(MangleType::OptimalSize),
Expand Down
2 changes: 1 addition & 1 deletion crates/next-core/src/next_config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1766,7 +1766,7 @@ impl NextConfig {

/// Returns the suffix to use for chunk loading.
#[turbo_tasks::function]
pub async fn chunk_suffix_path(self: Vc<Self>) -> Result<Vc<Option<RcStr>>> {
pub async fn asset_suffix_path(self: Vc<Self>) -> Result<Vc<Option<RcStr>>> {
let this = self.await?;

match &this.deployment_id {
Expand Down
10 changes: 8 additions & 2 deletions crates/next-core/src/next_edge/context.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ use turbo_tasks_fs::FileSystemPath;
use turbopack_browser::BrowserChunkingContext;
use turbopack_core::{
chunk::{
ChunkingConfig, ChunkingContext, MangleType, MinifyType, SourceMapsType, UnusedReferences,
chunk_id_strategy::ModuleIdStrategy,
AssetSuffix, ChunkingConfig, ChunkingContext, MangleType, MinifyType, SourceMapsType,
UnusedReferences, UrlBehavior, chunk_id_strategy::ModuleIdStrategy,
},
compile_time_info::{CompileTimeDefines, CompileTimeInfo, FreeVarReference, FreeVarReferences},
environment::{EdgeWorkerEnvironment, Environment, ExecutionEnvironment, NodeJsVersion},
Expand Down Expand Up @@ -324,6 +324,12 @@ pub async fn get_edge_chunking_context(
.client_roots_override(rcstr!("client"), client_root.clone())
.asset_root_path_override(rcstr!("client"), client_root.join("static/media")?)
.asset_base_path_override(rcstr!("client"), asset_prefix)
.url_behavior_override(
rcstr!("client"),
UrlBehavior {
suffix: AssetSuffix::FromGlobal(rcstr!("NEXT_CLIENT_ASSET_SUFFIX")),
},
)
// Since one can't read files in edge directly, any asset need to be fetched
// instead. This special blob url is handled by the custom fetch
// implementation in the edge sandbox. It will respond with the
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -171,8 +171,8 @@ async fn build_manifest(
let runtime_server_deployment_id_available =
*next_config.runtime_server_deployment_id_available().await?;
let suffix_path = if !runtime_server_deployment_id_available {
let chunk_suffix_path = next_config.chunk_suffix_path().owned().await?;
chunk_suffix_path.unwrap_or_default()
let asset_suffix_path = next_config.asset_suffix_path().owned().await?;
asset_suffix_path.unwrap_or_default()
} else {
rcstr!("")
};
Expand Down
45 changes: 38 additions & 7 deletions crates/next-core/src/next_server/context.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ use turbopack::{
};
use turbopack_core::{
chunk::{
ChunkingConfig, MangleType, MinifyType, SourceMapSourceType, SourceMapsType,
UnusedReferences, chunk_id_strategy::ModuleIdStrategy,
AssetSuffix, ChunkingConfig, MangleType, MinifyType, SourceMapSourceType, SourceMapsType,
UnusedReferences, UrlBehavior, chunk_id_strategy::ModuleIdStrategy,
},
compile_time_defines,
compile_time_info::{CompileTimeDefines, CompileTimeInfo, FreeVarReferences},
Expand Down Expand Up @@ -632,20 +632,21 @@ pub async fn get_server_module_options_context(

foreign_next_server_rules.extend(internal_custom_rules);

let url_rewrite_behavior = Some(
let (url_rewrite_behavior, static_url_tag) = {
//https://github.com/vercel/next.js/blob/bbb730e5ef10115ed76434f250379f6f53efe998/packages/next/src/build/webpack-config.ts#L1384
if let ServerContextType::PagesApi { .. } = ty {
UrlRewriteBehavior::Full
(Some(UrlRewriteBehavior::Full), None)
} else {
UrlRewriteBehavior::Relative
},
);
(Some(UrlRewriteBehavior::Relative), Some(rcstr!("client")))
}
};

let module_options_context = ModuleOptionsContext {
ecmascript: EcmascriptOptionsContext {
esm_url_rewrite_behavior: url_rewrite_behavior,
..module_options_context.ecmascript
},
static_url_tag,
..module_options_context
};

Expand Down Expand Up @@ -704,6 +705,15 @@ pub async fn get_server_module_options_context(
);
next_server_rules.extend(source_transform_rules);

let module_options_context = ModuleOptionsContext {
ecmascript: EcmascriptOptionsContext {
esm_url_rewrite_behavior: Some(UrlRewriteBehavior::Relative),
..module_options_context.ecmascript
},
static_url_tag: Some(rcstr!("client")),
..module_options_context
};

let foreign_code_module_options_context = ModuleOptionsContext {
module_rules: foreign_next_server_rules.clone(),
enable_webpack_loaders: foreign_enable_webpack_loaders,
Expand Down Expand Up @@ -777,6 +787,15 @@ pub async fn get_server_module_options_context(

next_server_rules.extend(source_transform_rules);

let module_options_context = ModuleOptionsContext {
ecmascript: EcmascriptOptionsContext {
esm_url_rewrite_behavior: Some(UrlRewriteBehavior::Relative),
..module_options_context.ecmascript
},
static_url_tag: Some(rcstr!("client")),
..module_options_context
};

let foreign_code_module_options_context = ModuleOptionsContext {
module_rules: foreign_next_server_rules.clone(),
enable_webpack_loaders: foreign_enable_webpack_loaders,
Expand Down Expand Up @@ -1044,6 +1063,12 @@ pub async fn get_server_chunking_context_with_client_assets(
next_mode.runtime_type(),
)
.asset_prefix(Some(asset_prefix))
.url_behavior_override(
rcstr!("client"),
UrlBehavior {
suffix: AssetSuffix::FromGlobal(rcstr!("NEXT_CLIENT_ASSET_SUFFIX")),
},
)
.minify_type(if *minify.await? {
MinifyType::Minify {
// React needs deterministic function names to work correctly.
Expand Down Expand Up @@ -1129,6 +1154,12 @@ pub async fn get_server_chunking_context(
.client_roots_override(rcstr!("client"), client_root.clone())
.asset_root_path_override(rcstr!("client"), client_root.join("static/media")?)
.asset_prefix_override(rcstr!("client"), asset_prefix)
.url_behavior_override(
rcstr!("client"),
UrlBehavior {
suffix: AssetSuffix::FromGlobal(rcstr!("NEXT_CLIENT_ASSET_SUFFIX")),
},
)
.minify_type(if *minify.await? {
MinifyType::Minify {
mangle: (!*no_mangling.await?).then_some(MangleType::OptimalSize),
Expand Down
2 changes: 1 addition & 1 deletion lerna.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,5 @@
"registry": "https://registry.npmjs.org/"
}
},
"version": "16.2.0-canary.5"
"version": "16.2.0-canary.6"
}
Loading
Loading