fix: QA audit fixes for Ralph-generated benchmark tasks

Critical fixes across 7 suites (crossrepo, docgen, investigation, nlqa, onboarding, security):
- Replace fabricated git SHAs with real ones in 3 crossrepo Dockerfiles (verified via GitHub API)
- Fix 9/10 wrong file paths in crossrepo-sym-003 ground_truth.json
- Fix hardcoded host paths in docgen-migration test.sh (use /tests/ Harbor path)
- Rewrite docgen-migration test.sh to follow Harbor verifier conventions (reward.txt, exit 0)
- Enrich near-empty docgen-api-003 task.toml with name/description/metadata
- Remove MCP/Sourcegraph tool name contamination from 5 suite CLAUDE.md files
- Remove MCP tool references from 5 instruction.md files (baseline contamination)
- chmod +x on 15 test.sh files missing execute permission

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Author: LoCoBench Bot

benchmarks/ccb_crossrepo/crossrepo-chain-001/instruction.md

@@ -60,6 +60,6 @@ Write your results to `/workspace/chain.json`:
 ## Notes
 
 - The kubernetes/kubernetes repository contains a staging directory (`staging/src/k8s.io/`) with code that is synced to separate repositories (kubernetes/api, kubernetes/apimachinery). For this task, treat them as separate codebases.
-- Use `go_to_definition` or cross-file search to trace imports and type references.
+- Use cross-file search or definition lookup to trace imports and type references.
 - You may encounter intermediate re-exports—document all steps.
 - Line numbers are approximate; +/- 50 lines is acceptable if the symbol is in that region.

benchmarks/ccb_crossrepo/crossrepo-chain-001/tests/test.sh

@@ -60,6 +60,6 @@ Write your results to `/workspace/chain.json`:
 ## Notes
 
 - The go-control-plane repository contains **generated code** from protobuf definitions. The `.pb.go` files are auto-generated from `.proto` files in data-plane-api.
-- Use `go_to_definition` or import tracing to navigate from Istio's usage to the generated Go types.
+- Use definition lookup or import tracing to navigate from Istio's usage to the generated Go types.
 - The final step should identify the **source `.proto` file**, not just the generated code.
 - Line numbers are approximate; +/- 50 lines is acceptable if the symbol is in that region.

benchmarks/ccb_crossrepo/crossrepo-chain-002/instruction.md

@@ -13,15 +13,15 @@ RUN apt-get update -qq && apt-get install -y -qq \
 
 WORKDIR /workspace
 
-# Clone kubernetes/api at pinned SHA (2026-02-16)
+# Clone kubernetes/api (same pinned SHA as crossrepo-chain-001)
 RUN git clone --depth 1 https://github.com/kubernetes/api.git api \
-    && cd api && git fetch --depth 1 origin c4a8a38b3ae0f0f8d6c8e0b3f7d9c8e8f9e0c8d7 \
-    && git checkout c4a8a38b3ae0f0f8d6c8e0b3f7d9c8e8f9e0c8d7
+    && cd api && git fetch --depth 1 origin f32ed1d60cf0787a512bebd6c06a4b84ae0b7cc7 \
+    && git checkout f32ed1d60cf0787a512bebd6c06a4b84ae0b7cc7
 
-# Clone kubernetes/apimachinery at pinned SHA (2026-02-16)
+# Clone kubernetes/apimachinery (same pinned SHA as crossrepo-chain-001)
 RUN git clone --depth 1 https://github.com/kubernetes/apimachinery.git apimachinery \
-    && cd apimachinery && git fetch --depth 1 origin a8e3f1c7b9d2e4f6a8b0c9d7e5f3a1b8c6d4e2f0 \
-    && git checkout a8e3f1c7b9d2e4f6a8b0c9d7e5f3a1b8c6d4e2f0
+    && cd apimachinery && git fetch --depth 1 origin b2e9f88ff6d4c50c13061a53b1239c7707354eda \
+    && git checkout b2e9f88ff6d4c50c13061a53b1239c7707354eda
 
 RUN mkdir -p /logs/verifier /logs/agent
 

benchmarks/ccb_crossrepo/crossrepo-chain-002/tests/test.sh

@@ -13,20 +13,20 @@ RUN apt-get update -qq && apt-get install -y -qq \
 
 WORKDIR /workspace
 
-# Clone envoyproxy/go-control-plane at pinned SHA (2026-02-16)
+# Clone envoyproxy/go-control-plane at xdsmatcher/v0.14.0
 RUN git clone --depth 1 https://github.com/envoyproxy/go-control-plane.git go-control-plane \
-    && cd go-control-plane && git fetch --depth 1 origin 0b8a46e8b5e5e8c5d9a7f4b3c2e1a8d9f6c7e5a4 \
-    && git checkout 0b8a46e8b5e5e8c5d9a7f4b3c2e1a8d9f6c7e5a4
+    && cd go-control-plane && git fetch --depth 1 origin 71637ad69bbc5f51fbb2562e612a4365292804a5 \
+    && git checkout 71637ad69bbc5f51fbb2562e612a4365292804a5
 
-# Clone istio/istio at pinned SHA (2026-02-16)
+# Clone istio/istio at 1.29.0
 RUN git clone --depth 1 https://github.com/istio/istio.git istio \
-    && cd istio && git fetch --depth 1 origin 7f8c6b5a4d3e2c1b9a8f7e6d5c4b3a2e1d0c9b8a \
-    && git checkout 7f8c6b5a4d3e2c1b9a8f7e6d5c4b3a2e1d0c9b8a
+    && cd istio && git fetch --depth 1 origin 2300e2458ab713c2c514a58da1ea8b03343ada7e \
+    && git checkout 2300e2458ab713c2c514a58da1ea8b03343ada7e
 
-# Clone emissary-ingress/emissary at pinned SHA (2026-02-16)
+# Clone emissary-ingress/emissary at v4.0.0-rc.1
 RUN git clone --depth 1 https://github.com/emissary-ingress/emissary.git emissary \
-    && cd emissary && git fetch --depth 1 origin 3e5a9c8b7d6f4a2e1c0b9a8d7f6e5c4b3a2d1e0f \
-    && git checkout 3e5a9c8b7d6f4a2e1c0b9a8d7f6e5c4b3a2d1e0f
+    && cd emissary && git fetch --depth 1 origin 3bbdbe0fafcc9dd6b9f54935d34f6614afb49302 \
+    && git checkout 3bbdbe0fafcc9dd6b9f54935d34f6614afb49302
 
 RUN mkdir -p /logs/verifier /logs/agent
 

benchmarks/ccb_crossrepo/crossrepo-impl-001/environment/Dockerfile

@@ -13,17 +13,15 @@ RUN apt-get update -qq && apt-get install -y -qq \
 
 WORKDIR /workspace
 
-# Clone hashicorp/terraform at pinned SHA (v1.10.5 release - 2026-02-14)
-# This version includes providers.Interface with all modern provider methods
+# Clone hashicorp/terraform at v1.15.0-alpha20260204
 RUN git clone --depth 1 https://github.com/hashicorp/terraform.git terraform \
-    && cd terraform && git fetch --depth 1 origin 6b7e0f87f5a7a2c8faa83d2c7a5f2f2e7d5c4b3a \
-    && git checkout 6b7e0f87f5a7a2c8faa83d2c7a5f2f2e7d5c4b3a
+    && cd terraform && git fetch --depth 1 origin f65c52c8995a627e64c82827c75edf99a9399e4b \
+    && git checkout f65c52c8995a627e64c82827c75edf99a9399e4b
 
-# Clone hashicorp/terraform-provider-aws at pinned SHA (v5.87.0 - 2026-02-13)
-# Recent stable release with full Provider interface implementation
+# Clone hashicorp/terraform-provider-aws at v6.32.1
 RUN git clone --depth 1 https://github.com/hashicorp/terraform-provider-aws.git terraform-provider-aws \
-    && cd terraform-provider-aws && git fetch --depth 1 origin 5a2c7f3e8b9d1a4f6e2c8d7b3a9f1e5c4b8d2a6f \
-    && git checkout 5a2c7f3e8b9d1a4f6e2c8d7b3a9f1e5c4b8d2a6f
+    && cd terraform-provider-aws && git fetch --depth 1 origin e9b4629e752e76551c7bca0f317e98e43bc96a7c \
+    && git checkout e9b4629e752e76551c7bca0f317e98e43bc96a7c
 
 RUN mkdir -p /logs/verifier /logs/agent
 

benchmarks/ccb_crossrepo/crossrepo-impl-002/environment/Dockerfile

@@ -3,47 +3,47 @@
   "entries": [
     {
       "repo": "hashicorp/terraform",
-      "file": "terraform/provider_mock.go",
+      "file": "internal/providers/testing/provider_mock.go",
       "function": "MockProvider"
     },
     {
       "repo": "hashicorp/terraform",
-      "file": "terraform/eval_provider.go",
+      "file": "internal/terraform/eval_provider.go",
       "function": "getProvider"
     },
     {
       "repo": "hashicorp/terraform",
-      "file": "terraform/node_provider_abstract.go",
-      "function": "NodeAbstractProvider.Execute"
+      "file": "internal/terraform/node_provider.go",
+      "function": "NodeApplyableProvider.Execute"
     },
     {
       "repo": "hashicorp/terraform",
-      "file": "terraform/node_resource_apply_instance.go",
+      "file": "internal/terraform/node_resource_apply_instance.go",
       "function": "managedResourceExecute"
     },
     {
       "repo": "hashicorp/terraform",
-      "file": "terraform/node_resource_plan_instance.go",
+      "file": "internal/terraform/node_resource_abstract_instance.go",
       "function": "plan"
     },
     {
       "repo": "hashicorp/terraform",
-      "file": "terraform/node_resource_plan_instance.go",
+      "file": "internal/terraform/node_resource_abstract_instance.go",
       "function": "refresh"
     },
     {
       "repo": "hashicorp/terraform",
-      "file": "terraform/node_resource_validate.go",
+      "file": "internal/terraform/node_resource_validate.go",
       "function": "validateResource"
     },
     {
       "repo": "hashicorp/terraform",
-      "file": "terraform/upgrade_resource_state.go",
+      "file": "internal/terraform/upgrade_resource_state.go",
       "function": "upgradeResourceState"
     },
     {
       "repo": "hashicorp/terraform-provider-aws",
-      "file": "internal/provider/provider.go",
+      "file": "internal/provider/factory.go",
       "function": "ProtoV5ProviderServerFactory"
     },
     {

benchmarks/ccb_crossrepo/crossrepo-sym-001/tests/test.sh

@@ -4,15 +4,14 @@ This suite tests your ability to generate accurate, comprehensive documentation
 
 ## Search Strategy
 
-**This repository is large.** You MUST use Sourcegraph MCP tools for documentation:
-
-- Use `keyword_search` to find exported functions, public APIs, and type definitions
-- Use `find_references` to understand how APIs are used in practice (for usage examples)
-- Use `go_to_definition` to read full function signatures, parameter types, and return types
-- Use `nls_search` for conceptual queries like "error handling patterns" or "configuration options"
-- Use `read_file` to examine existing documentation, docstrings, and comments
-- Use `list_files` to discover module structure and identify what needs documenting
-- Use `deepsearch` to understand complex subsystems before documenting them
+**This repository is large.** Use code search tools to efficiently navigate:
+
+- Search for exported functions, public APIs, and type definitions by keyword
+- Trace how APIs are used in practice via references (for usage examples)
+- Navigate to definitions to read full function signatures, parameter types, and return types
+- Use semantic search for conceptual queries like "error handling patterns" or "configuration options"
+- Read existing documentation, docstrings, and comments for context
+- Explore module structure to identify what needs documenting
 
 ## Output Requirements