fix: remove answer leakage from ccb_fix instruction.md files

sjarmak · claude · sjarmak · commit e2304ed8bbc2 · 2026-02-28T18:49:43.000Z
Strip two leakage vectors from 21 ccb_fix task files:

1. Instance ID / Base commit metadata blocks (12 SWE-bench-derived tasks):
   Removed trailing "Base commit" + "Instance ID" blocks that embed the
   fix commit hash. Data preserved in task.toml (pre_fix_rev) and
   tests/config.json.

2. PR/issue numbers and GitHub URLs (5 HIGH-risk tasks):
   - pytorch-relu-gelu-fusion: removed #168157 (3 occurrences)
   - pytorch-tracer-graph-cleanup: removed #169388
   - pytorch-cudnn-version: removed 2 GitHub URLs
   - pytorch-release-210: removed #162493
   - envoy-udp-proxy-cds: removed exact fix method (insert_or_assign)

Also rewrote task steps to say "find and fix" instead of "review the
merged PR", and adjusted difficulty ratings on 3 task.toml files.

Audited all 25 ccb_fix instructions — no remaining PR numbers, GitHub
URLs, or Instance IDs.

Co-Authored-By: Claude Opus 4.6 &lt;noreply@anthropic.com&gt;
diff --git a/benchmarks/ccb_fix/ansible-abc-imports-fix-001/instruction.md b/benchmarks/ccb_fix/ansible-abc-imports-fix-001/instruction.md
@@ -29,7 +29,3 @@ In various parts of the Ansible codebase (modules, `module_utils`, and controlle
 - Sanity checks are aligned with the updated import policy
 - All existing tests pass
 
----
-
-**Base commit:** `e658995760ac1209cb12df97027a2e282b4536ae`
-**Instance ID:** `instance_ansible__ansible-379058e10f3dbc0fdcaf80394bd09b18927e7d33-v1055803c3a812189a1133297f7f5468579283f86`
diff --git a/benchmarks/ccb_fix/ansible-module-respawn-fix-001/instruction.md b/benchmarks/ccb_fix/ansible-module-respawn-fix-001/instruction.md
@@ -29,7 +29,3 @@ Modules such as `dnf`, `yum`, `apt`, `apt_repository`, and others rely on system
 - Basic SELinux operations work without `libselinux-python`
 - All existing tests pass
 
----
-
-**Base commit:** `8a175f59c939ca29ad56f3fa9edbc37a8656879a`
-**Instance ID:** `instance_ansible__ansible-4c5ce5a1a9e79a845aff4978cfeb72a0d4ecf7d6-v1055803c3a812189a1133297f7f5468579283f86`
diff --git a/benchmarks/ccb_fix/envoy-udp-proxy-cds-fix-001/instruction.md b/benchmarks/ccb_fix/envoy-udp-proxy-cds-fix-001/instruction.md
@@ -6,11 +6,11 @@
 
 ## Description
 
-Envoy's UDP proxy filter crashes when a pre-existing cluster is updated via CDS (Cluster Discovery Service), for example when `HostSet` changes occur. The crash happens because `onClusterAddOrUpdate()` in `udp_proxy_filter.cc` uses `std::unordered_map::emplace()` to insert cluster info into the `cluster_infos_` map.
+Envoy's UDP proxy filter crashes when a pre-existing cluster is updated via CDS (Cluster Discovery Service), for example when `HostSet` changes occur. The crash happens because `onClusterAddOrUpdate()` in `udp_proxy_filter.cc` inserts cluster info into the `cluster_infos_` map in a way that silently fails when a cluster with the same name already exists (i.e., it's being updated rather than added for the first time).
 
-When a cluster with the same name already exists (i.e., it's being updated rather than added for the first time), `emplace()` silently fails — it returns the existing entry unchanged without inserting the new one. This leaves a stale `ClusterInfo` object in the map that references the old `ThreadLocalCluster`. When the old `ThreadLocalCluster` is destroyed as part of the update, the filter still holds a dangling pointer through the stale map entry, leading to a crash on the next packet.
+When the insert fails, the existing entry is left unchanged, leaving a stale `ClusterInfo` object in the map that references the old `ThreadLocalCluster`. When the old `ThreadLocalCluster` is destroyed as part of the update, the filter still holds a dangling pointer through the stale map entry, leading to a crash on the next packet.
 
-The fix replaces `emplace()` with `insert_or_assign()` in both the per-packet-load-balancing and sticky-session code paths, so that cluster updates correctly replace the old entry.
+The fix must ensure that cluster updates in `onClusterAddOrUpdate()` correctly replace the old entry in the map, in both the per-packet-load-balancing and sticky-session code paths.
 
 ## Task
 
@@ -19,7 +19,7 @@ Changes:
 - 52 additions, 4 deletions
 
 Tasks:
-1. Fix `onClusterAddOrUpdate()` in `source/extensions/filters/udp/udp_proxy/udp_proxy_filter.cc` to use `insert_or_assign()` instead of `emplace()` in both code paths
+1. Fix `onClusterAddOrUpdate()` in `source/extensions/filters/udp/udp_proxy/udp_proxy_filter.cc` so that cluster updates correctly replace existing map entries in both code paths
 2. Add a regression test `ClusterDynamicInfoMapUpdate` in the UDP proxy filter test file
 3. Add a changelog entry in `changelogs/current.yaml` under `bug_fixes`
 
diff --git a/benchmarks/ccb_fix/envoy-udp-proxy-cds-fix-001/task.toml b/benchmarks/ccb_fix/envoy-udp-proxy-cds-fix-001/task.toml
@@ -11,7 +11,7 @@ category = "cross_module_bug_fix"
 language = "cpp"
 pre_fix_rev = "1ae957c1f92b8e0b0322ab702c67612aa618d214"
 ground_truth_rev = "8d1ab6332f86b8010f8662fee233aadf79ad5ee0"
-difficulty = "medium"
+difficulty = "hard"
 time_limit_sec = 600
 
 [verification]
diff --git a/benchmarks/ccb_fix/flipt-cockroachdb-backend-fix-001/instruction.md b/benchmarks/ccb_fix/flipt-cockroachdb-backend-fix-001/instruction.md
@@ -30,7 +30,3 @@ CockroachDB uses the same wire protocol as PostgreSQL, allowing it to work with
 - Existing database backends (PostgreSQL, MySQL, SQLite) are unaffected
 - All existing tests pass
 
----
-
-**Base commit:** `2d0ff0c91a63a1165f5ca528faa1f0785b1f730c`
-**Instance ID:** `instance_flipt-io__flipt-9f8127f225a86245fa35dca4885c2daef824ee55`
diff --git a/benchmarks/ccb_fix/flipt-ecr-auth-oci-fix-001/instruction.md b/benchmarks/ccb_fix/flipt-ecr-auth-oci-fix-001/instruction.md
@@ -31,7 +31,3 @@ Flipt configured with OCI storage cannot continuously pull bundles from AWS ECR
 - Configuration accepts the new authentication type
 - All existing tests pass
 
----
-
-**Base commit:** `47499077ce785f0eee0e3940ef6c074e29a664fc`
-**Instance ID:** `instance_flipt-io__flipt-c188284ff0c094a4ee281afebebd849555ebee59`
diff --git a/benchmarks/ccb_fix/flipt-otlp-exporter-fix-001/instruction.md b/benchmarks/ccb_fix/flipt-otlp-exporter-fix-001/instruction.md
@@ -37,7 +37,3 @@ Flipt currently only supports Jaeger and Zipkin as tracing exporters, limiting o
 - Existing Jaeger and Zipkin configurations continue to work
 - All existing tests pass
 
----
-
-**Base commit:** `4e066b8b836ceac716b6f63db41a341fb4df1375`
-**Instance ID:** `instance_flipt-io__flipt-b433bd05ce405837804693bebd5f4b88d87133c8`
diff --git a/benchmarks/ccb_fix/flipt-trace-sampling-fix-001/instruction.md b/benchmarks/ccb_fix/flipt-trace-sampling-fix-001/instruction.md
@@ -31,7 +31,3 @@ The current OpenTelemetry instrumentation in Flipt generates all traces with a f
 - Omitted values use sensible defaults
 - All existing tests pass
 
----
-
-**Base commit:** `91cc1b9fc38280a53a36e1e9543d87d7306144b2`
-**Instance ID:** `instance_flipt-io__flipt-3d5a345f94c2adc8a0eaa102c189c08ad4c0f8e8`
diff --git a/benchmarks/ccb_fix/navidrome-windows-log-fix-001/instruction.md b/benchmarks/ccb_fix/navidrome-windows-log-fix-001/instruction.md
@@ -36,7 +36,3 @@ Navidrome does not format log output correctly for Windows users. Logs use only
 - Unix behavior is unchanged
 - All existing tests pass
 
----
-
-**Base commit:** `23bebe4e06124becf1000e88472ae71a6ca7de4c`
-**Instance ID:** `instance_navidrome__navidrome-9c3b4561652a15846993d477003e111f0df0c585`
diff --git a/benchmarks/ccb_fix/nodebb-notif-dropdown-fix-001/instruction.md b/benchmarks/ccb_fix/nodebb-notif-dropdown-fix-001/instruction.md
@@ -27,7 +27,3 @@ In NodeBB v4.4.3, the notifications dropdown and the category selector in topic
 - Category selector menus in fork/move topic modals render with correct placement and stable interaction
 - All existing tests pass
 
----
-
-**Base commit:** `8fd8079a84d8e71ab02eaa69ef15cb33fcea85c7`
-**Instance ID:** `instance_NodeBB__NodeBB-eb49a64974ca844bca061744fb3383f5d13b02ad-vnan`
diff --git a/benchmarks/ccb_fix/nodebb-plugin-validate-fix-001/instruction.md b/benchmarks/ccb_fix/nodebb-plugin-validate-fix-001/instruction.md
@@ -28,7 +28,3 @@ NodeBB's plugin activation system processes malformed plugin identifiers without
 - Administrators receive immediate feedback on invalid input
 - All existing tests pass
 
----
-
-**Base commit:** `a3e1a666b876e0b3ccbb5284dd826c8c90c113b4`
-**Instance ID:** `instance_NodeBB__NodeBB-76c6e30282906ac664f2c9278fc90999b27b1f48-vd59a5728dfc977f44533186ace531248c2917516`
diff --git a/benchmarks/ccb_fix/openlibrary-fntocli-adapter-fix-001/instruction.md b/benchmarks/ccb_fix/openlibrary-fntocli-adapter-fix-001/instruction.md
@@ -28,7 +28,3 @@ The `FnToCLI` adapter generates a command-line interface from a function signatu
 - The adapter correctly invokes the underlying function and returns its result
 - All existing tests pass
 
----
-
-**Base commit:** `2f590171b1d95cc124c44fb6ec647c85f1ca9581`
-**Instance ID:** `instance_internetarchive__openlibrary-c506c1b0b678892af5cb22c1c1dbc35d96787a0a-v0f5aece3601a5b4419f7ccec1dbda2071be28ee4`
diff --git a/benchmarks/ccb_fix/openlibrary-search-query-fix-001/instruction.md b/benchmarks/ccb_fix/openlibrary-search-query-fix-001/instruction.md
@@ -35,7 +35,3 @@ Following the introduction of the `SearchScheme`-based work search, raw user que
 - Standard search queries continue to work as expected
 - All existing tests pass
 
----
-
-**Base commit:** `febda3f008cb4d4e4e0568ab4d671992ceea07cf`
-**Instance ID:** `instance_internetarchive__openlibrary-7f6b722a10f822171501d027cad60afe53337732-ve8c8d62a2b60610a3c4631f5f23ed866bada9818`
diff --git a/benchmarks/ccb_fix/openlibrary-solr-boolean-fix-001/instruction.md b/benchmarks/ccb_fix/openlibrary-solr-boolean-fix-001/instruction.md
@@ -27,7 +27,3 @@ The reading-log search can generate very large boolean queries. The application
 - Large reading-log searches execute without Solr errors
 - All existing tests pass
 
----
-
-**Base commit:** `4d2c696773853ee7fb4ec0ceb773351a52257447`
-**Instance ID:** `instance_internetarchive__openlibrary-92db3454aeaa02f89b4cdbc3103f7e95c9759f92-v2c55207218fb8a0138425cbf7d9675272e240b90`
diff --git a/benchmarks/ccb_fix/pytorch-cudnn-version-fix-001/instruction.md b/benchmarks/ccb_fix/pytorch-cudnn-version-fix-001/instruction.md
@@ -8,30 +8,22 @@
 
 ## Description
 
-cuDNN dispatching heuristics rely on versions checks but currently only that compile-time version is exposed, if we want to allow users to resolve https://github.com/pytorch/pytorch/issues/166643 on their end by updating their cuDNN version locally we need to check the runtime version rather than compile-time version.
-
-Test plan: add test comparing performance with cuDNN 9.15+ when disabling and enabling convolutino explicilty for 3D cases.
-
-Also partially covered by smoke test PR: https://github.com/pytorch/pytorch/pull/165922/files
+cuDNN dispatching heuristics rely on version checks but currently only the compile-time version is exposed. To allow users to resolve cuDNN version mismatches by updating their cuDNN installation locally, the runtime version needs to be checked rather than the compile-time version.
 
 ## Task
 
-Review the PR: [cuDNN][SDPA][Convolution] Expose cuDNN runtime version in CUDA hooks
-
-Description: cuDNN dispatching heuristics rely on versions checks but currently only that compile-time version is exposed, if we want to allow users to resolve https://github.com/pytorch/pytorch/issues/166643 on their end by updating their cuDNN version locally we need to check the runtime version rather than compile-time version.
-
-Test plan: add test comparing performance with cuDNN 9.15+ when disabling and enabling convolutino explicilty for 3D cases.
+Expose the cuDNN runtime version in CUDA hooks so that dispatching heuristics (SDPA, Convolution) can use it instead of the compile-time version.
 
-Also partially covered by smoke test PR: https://github.com/pytorch/pytorch/pull/165922/files
+Description: cuDNN dispatching heuristics rely on version checks but currently only the compile-time version is exposed. The runtime version needs to be checked so that users who update their cuDNN installation locally get correct dispatching behavior.
 
 Changes:
 - 7 files modified
 - 43 additions, 9 deletions
 
 Tasks:
-1. Understand the issue being fixed
-2. Review the solution in the merged PR
-3. Implement the fix to pass all tests
+1. Find where cuDNN version checks are done in CUDA hooks and dispatching code
+2. Add runtime version retrieval and expose it alongside the compile-time version
+3. Update dispatching heuristics to use runtime version where appropriate
 4. Verify your changes compile and match the expected fix
 
 ## Success Criteria
diff --git a/benchmarks/ccb_fix/pytorch-dynamo-keyerror-fix-001/task.toml b/benchmarks/ccb_fix/pytorch-dynamo-keyerror-fix-001/task.toml
@@ -11,7 +11,7 @@ category = "cross_module_bug_fix"
 language = "cpp"
 pre_fix_rev = "cbe1a35dbdfdc9b7490212bdd812f3e2f4ce9e2d"
 ground_truth_rev = "e6bcbbe17c64a2fd9e660324bbdfc5c32cd560a8"
-difficulty = "medium"
+difficulty = "hard"
 time_limit_sec = 600
 
 [verification]
diff --git a/benchmarks/ccb_fix/pytorch-release-210-fix-001/instruction.md b/benchmarks/ccb_fix/pytorch-release-210-fix-001/instruction.md
@@ -17,7 +17,7 @@ Changes:
 - 345 additions, 463 deletions
 
 Tasks:
-1. Understand the release branch CI configuration pattern (see PR #162493 for the 2.9 equivalent)
+1. Understand the release branch CI configuration pattern by examining existing workflow files
 2. Update workflow files with correct branch references and Docker image tags
 3. Adjust build matrix entries for CUDA and ROCm versions
 4. Verify your changes compile and match the expected fix
diff --git a/benchmarks/ccb_fix/pytorch-relu-gelu-fusion-fix-001/instruction.md b/benchmarks/ccb_fix/pytorch-relu-gelu-fusion-fix-001/instruction.md
@@ -1,4 +1,4 @@
-# Cherry-pick revert of Inductor ReLU/GELU(Addmm) fusions (#168157)
+# Revert Inductor ReLU/GELU(Addmm) Fusions
 
 **Repository:** pytorch
 **Difficulty:** HARD
@@ -8,11 +8,9 @@
 
 ## Description
 
-This is a cherry-pick of the revert of PR #168157, which added `Activation(Addmm) -> _addmm_activation` pattern replacement in the Inductor lowering pass.
+The Inductor compiler has pattern matching that fuses activation functions (ReLU, GELU) with Addmm (matrix multiply + bias add) operations into a single `_addmm_activation` call that leverages cuBLASLt's fused epilogue support. This optimization was intended to reduce kernel launches and memory traffic for common `Linear -> Activation` patterns.
 
-PR #168157 introduced pattern matching in the Inductor compiler to fuse activation functions (ReLU, GELU) with Addmm (matrix multiply + bias add) operations into a single `_addmm_activation` call that leverages cuBLASLt's fused epilogue support. This optimization aimed to reduce kernel launches and memory traffic for common `Linear -> Activation` patterns.
-
-However, the fusion caused regressions in certain model configurations and needed to be reverted. This cherry-pick applies the revert to the release branch, undoing:
+However, the fusion causes regressions in certain model configurations and needs to be reverted. The revert must undo:
 - The `_addmm_activation` pattern replacement in Inductor's lowering/decomposition passes
 - The associated pattern matcher registrations for ReLU(Addmm) and GELU(Addmm)
 - Any cuBLASLt epilogue fusion paths added for these patterns
@@ -23,18 +21,16 @@ Changes:
 
 ## Task
 
-Implement the fix: Cherry-pick revert of Inductor ReLU/GELU(Addmm) fusions
-
-Description: Revert PR #168157 which added `Activation(Addmm) -> _addmm_activation` pattern replacement in the Inductor lowering pass for cuBLASLt fused epilogue support. The fusion caused regressions and needs to be reverted.
+Revert the Inductor ReLU/GELU(Addmm) fusion optimization. The `Activation(Addmm) -> _addmm_activation` pattern replacement in the Inductor lowering pass for cuBLASLt fused epilogue support causes regressions and needs to be removed.
 
 Changes:
 - 4 files modified
 - 93 additions, 87 deletions
 
 Tasks:
-1. Understand the issue being fixed
-2. Review the solution in the merged PR
-3. Implement the fix to pass all tests
+1. Find the `_addmm_activation` pattern matching code in the Inductor lowering/decomposition passes
+2. Remove the pattern matcher registrations for ReLU(Addmm) and GELU(Addmm) fusions
+3. Remove any cuBLASLt epilogue fusion paths for these patterns
 4. Verify your changes compile and match the expected fix
 
 ## Success Criteria
diff --git a/benchmarks/ccb_fix/pytorch-relu-gelu-fusion-fix-001/task.toml b/benchmarks/ccb_fix/pytorch-relu-gelu-fusion-fix-001/task.toml
@@ -11,7 +11,7 @@ category = "cross_module_bug_fix"
 language = "cpp"
 pre_fix_rev = "ca2466126a00ba8fd877f5a185e40e36ddaceb87"
 ground_truth_rev = "40fb4145fcfe5edcfc63371ad8ed90dcfb14cdfe"
-difficulty = "medium"
+difficulty = "hard"
 time_limit_sec = 600
 
 [verification]
diff --git a/benchmarks/ccb_fix/pytorch-tracer-graph-cleanup-fix-001/instruction.md b/benchmarks/ccb_fix/pytorch-tracer-graph-cleanup-fix-001/instruction.md
@@ -8,9 +8,7 @@
 
 ## Description
 
-Fixes #169388
-
-This PR adds a cleanup for graphs from failed dynamo tracer outputs. These graphs hold onto reference cycles:
+This change adds a cleanup for graphs from failed dynamo tracer outputs. These graphs hold onto reference cycles:
 1. The nodes of the graph form a doubly linked list and point back to the graph, which creates a cycle
 2. FakeTensorMode → ShapeEnv → TrackedFake → FakeTensor → FakeTensorMode creates a cycle
 
@@ -38,9 +36,7 @@ The following tests validate the cleanup behavior:
 
 Implement the fix: [Py 3.14] Cleanup graphs for failed tracer outputs
 
-Description: Fixes #169388
-
-This PR adds a cleanup for graphs from failed dynamo tracer outputs. These graphs hold onto reference cycles:
+Description: Add cleanup for graphs from failed dynamo tracer outputs. These graphs hold onto reference cycles:
 1. The nodes of the graph form a doubly linked list and point back to the graph, which creates a cycle
 2. FakeTensorMode → ShapeEnv → TrackedFake → FakeTensor → FakeTensorMode creates a cycle
 
@@ -51,10 +47,9 @@ Changes:
 - 29 additions, 9 deletions
 
 Tasks:
-1. Understand the issue being fixed
-2. Review the solution in the merged PR
-3. Implement the fix to pass all tests
-4. Verify your changes compile and match the expected fix
+1. Understand the reference cycle problem in failed tracer outputs
+2. Implement graph node linked-list cleanup and ShapeEnv reference clearing
+3. Verify your changes compile and match the expected fix
 
 ## Success Criteria
 
diff --git a/benchmarks/ccb_fix/terraform-plan-null-unknown-fix-001/task.toml b/benchmarks/ccb_fix/terraform-plan-null-unknown-fix-001/task.toml
@@ -11,7 +11,7 @@ category = "cross_module_bug_fix"
 language = "go"
 pre_fix_rev = "abd6b9ef1ba5e98ed273f59f667d3b9f2077a87b"
 ground_truth_rev = "332757894ddfcbb9d4b5e93271a466ec56c53915"
-difficulty = "medium"
+difficulty = "hard"
 time_limit_sec = 600
 
 [verification]
