Commit af3e69a
feat: oracle hydration, model fix, protonmail removal, MCP-unique task prep
- Fix critical model default in run_selected_tasks.sh: opus -> haiku
- Hydrate 47 MCP-unique task_spec.json oracles from oracle_answer.json
- Remove 3 protonmail tasks (unresolvable git apply --allow-empty verifier bug)
- Update task.toml timeouts for build/design tasks based on rerun analysis
- Add 4 new MCP-unique tasks (057, 042, 050, 091) with eval.sh verifiers
- Add direct_verifier.sh for 5 ccb_mcp_org tasks (081-083, 122, 127)
- Add batch selection configs: mcp_unique_batch, rerun_haiku_timeout, etc.
- Update MCP task instructions and metadata across all 10 ccb_mcp_* suites
- Add Makefile, analysis scripts, doc updates
- Update generate_manifest.py with MCP-unique suite detection
- Update selected_benchmark_tasks.json and config_utils.py
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>1 parent 8921894 commit af3e69a
File tree
295 files changed
+11177
-11446
lines changed- benchmarks
- ccb_build
- camel-fix-protocol-feat-001
- flink-pricing-window-feat-001
- k8s-noschedule-taint-feat-001
- k8s-score-normalizer-refac-001
- rust-subtype-relation-refac-001
- servo-scrollend-event-feat-001
- ccb_design
- camel-routing-arch-001
- etcd-grpc-api-upgrade-001
- flink-checkpoint-arch-001
- k8s-crd-lifecycle-arch-001
- k8s-dra-allocation-impact-001
- k8s-scheduler-arch-001
- kafka-flink-streaming-arch-001
- ccb_fix
- openlibrary-solr-boolean-fix-001
- protonmail-conv-testhooks-fix-001
- environment
- solution
- tests
- protonmail-dropdown-sizing-fix-001
- environment
- solution
- tests
- protonmail-holiday-calendar-fix-001
- environment
- solution
- tests
- ccb_mcp_compliance
- ccx-compliance-051
- tests
- ccx-compliance-052
- tests
- ccx-compliance-053
- tests
- ccx-compliance-057
- tests
- ccx-compliance-115
- tests
- ccx-compliance-118
- tests
- ccx-compliance-124
- ccb_mcp_crossorg
- ccx-crossorg-061
- tests
- ccx-crossorg-062
- tests
- ccx-crossorg-066
- ccx-crossorg-121
- ccx-crossorg-132
- ccb_mcp_crossrepo_tracing
- ccx-config-trace-003
- tests
- ccx-config-trace-010
- tests
- ccx-dep-trace-001
- tests
- ccx-dep-trace-002
- tests
- ccx-dep-trace-004
- tests
- ccx-dep-trace-102
- tests
- ccx-dep-trace-116
- tests
- ccx-dep-trace-123
- ccx-dep-trace-133
- ccb_mcp_crossrepo/ccx-dep-trace-106
- tests
- ccb_mcp_domain
- ccx-domain-071
- tests
- ccx-domain-072
- tests
- ccx-domain-073
- tests
- ccx-domain-074
- tests
- ccx-domain-101
- tests
- ccx-domain-112
- tests
- ccx-domain-120
- tests
- ccx-domain-129
- ccx-domain-137
- ccx-domain-140
- ccb_mcp_incident
- ccx-incident-031
- tests
- ccx-incident-032
- tests
- ccx-incident-033
- tests
- ccx-incident-034
- tests
- ccx-incident-037
- tests
- ccx-incident-108
- tests
- ccx-incident-110
- tests
- ccx-incident-113
- tests
- ccx-incident-125
- ccx-incident-131
- ccx-incident-139
- ccb_mcp_migration
- ccx-migration-022
- tests
- ccx-migration-025
- tests
- ccx-migration-026
- tests
- ccx-migration-027
- tests
- ccx-migration-107
- tests
- ccx-migration-114
- tests
- ccx-migration-117
- tests
- ccb_mcp_onboarding
- ccx-onboard-041
- tests
- ccx-onboard-042
- tests
- ccx-onboard-043
- tests
- ccx-onboard-044
- tests
- ccx-onboard-050
- tests
- ccx-onboard-103
- tests
- ccx-onboard-109
- tests
- ccx-onboard-128
- ccx-onboard-134
- ccx-onboard-136
- ccx-onboard-138
- ccb_mcp_org
- ccx-agentic-081
- tests
- ccx-agentic-082
- tests
- ccx-agentic-083
- tests
- ccx-agentic-122
- tests
- ccx-agentic-127
- tests
- ccb_mcp_platform
- ccx-platform-091
- tests
- ccx-platform-094
- tests
- ccx-platform-100
- tests
- ccx-platform-104
- tests
- ccx-platform-119
- tests
- ccb_mcp_security
- ccx-vuln-remed-011
- tests
- ccx-vuln-remed-012
- tests
- ccx-vuln-remed-013
- tests
- ccx-vuln-remed-014
- tests
- ccx-vuln-remed-105
- tests
- ccx-vuln-remed-111
- tests
- ccx-vuln-remed-126
- ccx-vuln-remed-130
- ccx-vuln-remed-135
- ccx-vuln-remed-141
- configs
- docs
- ops
- reference
- scripts
- ccb_metrics
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
295 files changed
+11177
-11446
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
| 47 | + | |
47 | 48 | | |
48 | 49 | | |
49 | 50 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
26 | | - | |
| 26 | + | |
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
26 | | - | |
| 26 | + | |
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
24 | | - | |
| 24 | + | |
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
26 | | - | |
| 26 | + | |
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
26 | | - | |
| 26 | + | |
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
24 | | - | |
| 24 | + | |
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
26 | | - | |
| 26 | + | |
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
22 | | - | |
| 22 | + | |
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
0 commit comments