Skip to content

Commit 5f831b6

Browse files
sjarmakclaude
sjarmak
and
claude
committed
feat: curate oracle_answer.json for ccx-agentic-223 through ccx-agentic-237
Add file-level oracle answers for 15 ccb_mcp_org agentic tasks covering: - ccx-agentic-223: Kubernetes kubelet pod lifecycle (PLEG, VolumeManager, DeviceManager, KubeRuntime) - ccx-agentic-224: Kubernetes CSI driver integration (csi_plugin, csi_client, csi_attacher, nodeinfomanager, pv_controller) - ccx-agentic-225: Kubernetes CNI network plugin interface (kuberuntime_sandbox, kuberuntime_manager, dns, cri-api/services) - ccx-agentic-226: Envoy Registry/FactoryRegistry pattern - ccx-agentic-227: Envoy stats TagProducer/StatNamePool - ccx-agentic-228: Envoy ListenerManager - ccx-agentic-229: Grafana unified alerting/ngalert - ccx-agentic-230: Grafana query caching - ccx-agentic-231: Grafana OrgUser/permissions - ccx-agentic-232: Kafka ReplicaFetcherThread/LeaderEpochFileCache - ccx-agentic-233: Kafka RangeAssignor/RoundRobinAssignor/StickyAssignor - ccx-agentic-234: Kafka OffsetAndMetadata/GroupCoordinator/OffsetFetch - ccx-agentic-235: etcd client gRPC retry/balancer - ccx-agentic-236: etcd MVCC compaction/watch stream - ccx-agentic-237: Prometheus WAL (tsdb/wlog) using sg-evals/prometheus--v3.2.1 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 2212726 commit 5f831b6

File tree

15 files changed

+255
-75
lines changed

15 files changed

+255
-75
lines changed

‎benchmarks/ccb_mcp_org/ccx-agentic-223/tests/oracle_answer.json‎

Lines changed: 20 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,23 @@
11
{
2-
"files": [],
3-
"text": "Oracle not yet curated \u2014 task stub.",
2+
"files": [
3+
"github.com/sg-evals/kubernetes--v1.32.0/pkg/kubelet/kubelet.go",
4+
"github.com/sg-evals/kubernetes--v1.32.0/pkg/kubelet/pleg/pleg.go",
5+
"github.com/sg-evals/kubernetes--v1.32.0/pkg/kubelet/pleg/generic.go",
6+
"github.com/sg-evals/kubernetes--v1.32.0/pkg/kubelet/pleg/evented.go",
7+
"github.com/sg-evals/kubernetes--v1.32.0/pkg/kubelet/volumemanager/volume_manager.go",
8+
"github.com/sg-evals/kubernetes--v1.32.0/pkg/kubelet/cm/devicemanager/manager.go",
9+
"github.com/sg-evals/kubernetes--v1.32.0/pkg/kubelet/cm/devicemanager/types.go",
10+
"github.com/sg-evals/kubernetes--v1.32.0/pkg/kubelet/kuberuntime/kuberuntime_manager.go"
11+
],
12+
"symbols": [],
13+
"repo_set_id": "kubernetes-ecosystem",
414
"_metadata": {
5-
"oracle_type": "tbd",
6-
"status": "pending_curation"
15+
"discovery_method": "sourcegraph_mcp_keyword_search",
16+
"timestamp": "2026-02-28T00:00:00Z",
17+
"repos_searched": [
18+
"sg-evals/kubernetes--v1.32.0",
19+
"sg-evals/cilium--ad6b298d",
20+
"sg-evals/etcd-io-etcd"
21+
]
722
}
8-
}
23+
}

‎benchmarks/ccb_mcp_org/ccx-agentic-224/tests/oracle_answer.json‎

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,21 @@
11
{
2-
"files": [],
3-
"text": "Oracle not yet curated \u2014 task stub.",
2+
"files": [
3+
"github.com/sg-evals/kubernetes--v1.32.0/pkg/volume/csi/csi_plugin.go",
4+
"github.com/sg-evals/kubernetes--v1.32.0/pkg/volume/csi/csi_client.go",
5+
"github.com/sg-evals/kubernetes--v1.32.0/pkg/volume/csi/csi_attacher.go",
6+
"github.com/sg-evals/kubernetes--v1.32.0/pkg/volume/csi/nodeinfomanager/nodeinfomanager.go",
7+
"github.com/sg-evals/kubernetes--v1.32.0/pkg/controller/volume/persistentvolume/pv_controller.go",
8+
"github.com/sg-evals/kubernetes--v1.32.0/pkg/controller/volume/persistentvolume/pv_controller_base.go"
9+
],
10+
"symbols": [],
11+
"repo_set_id": "kubernetes-ecosystem",
412
"_metadata": {
5-
"oracle_type": "tbd",
6-
"status": "pending_curation"
13+
"discovery_method": "sourcegraph_mcp_keyword_search",
14+
"timestamp": "2026-02-28T00:00:00Z",
15+
"repos_searched": [
16+
"sg-evals/kubernetes--v1.32.0",
17+
"sg-evals/cilium--ad6b298d",
18+
"sg-evals/etcd-io-etcd"
19+
]
720
}
8-
}
21+
}

‎benchmarks/ccb_mcp_org/ccx-agentic-225/tests/oracle_answer.json‎

Lines changed: 17 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,20 @@
11
{
2-
"files": [],
3-
"text": "Oracle not yet curated \u2014 task stub.",
2+
"files": [
3+
"github.com/sg-evals/kubernetes--v1.32.0/pkg/kubelet/kuberuntime/kuberuntime_sandbox.go",
4+
"github.com/sg-evals/kubernetes--v1.32.0/pkg/kubelet/kuberuntime/kuberuntime_manager.go",
5+
"github.com/sg-evals/kubernetes--v1.32.0/pkg/kubelet/network/dns/dns.go",
6+
"github.com/sg-evals/kubernetes--v1.32.0/staging/src/k8s.io/cri-api/pkg/apis/services.go",
7+
"github.com/sg-evals/kubernetes--v1.32.0/pkg/kubelet/kubelet.go"
8+
],
9+
"symbols": [],
10+
"repo_set_id": "kubernetes-ecosystem",
411
"_metadata": {
5-
"oracle_type": "tbd",
6-
"status": "pending_curation"
12+
"discovery_method": "sourcegraph_mcp_keyword_search",
13+
"timestamp": "2026-02-28T00:00:00Z",
14+
"repos_searched": [
15+
"sg-evals/kubernetes--v1.32.0",
16+
"sg-evals/cilium--ad6b298d",
17+
"sg-evals/etcd-io-etcd"
18+
]
719
}
8-
}
20+
}

‎benchmarks/ccb_mcp_org/ccx-agentic-226/tests/oracle_answer.json‎

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,17 @@
11
{
2-
"files": [],
3-
"text": "Oracle not yet curated \u2014 task stub.",
2+
"files": [
3+
"github.com/sg-evals/envoy--v1.31.2/envoy/registry/registry.h",
4+
"github.com/sg-evals/envoy--v1.31.2/test/common/config/registry_test.cc",
5+
"github.com/sg-evals/envoy--v1.31.2/source/common/network/resolver_impl.cc"
6+
],
7+
"symbols": [],
8+
"repo_set_id": "envoy-service-mesh",
49
"_metadata": {
5-
"oracle_type": "tbd",
6-
"status": "pending_curation"
10+
"discovery_method": "sourcegraph_mcp_keyword_search",
11+
"timestamp": "2026-02-28T00:00:00Z",
12+
"repos_searched": [
13+
"sg-evals/envoy--v1.31.2",
14+
"sg-evals/grpc--957dba5e"
15+
]
716
}
8-
}
17+
}

‎benchmarks/ccb_mcp_org/ccx-agentic-227/tests/oracle_answer.json‎

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,21 @@
11
{
2-
"files": [],
3-
"text": "Oracle not yet curated \u2014 task stub.",
2+
"files": [
3+
"github.com/sg-evals/envoy--v1.31.2/source/common/stats/tag_producer_impl.cc",
4+
"github.com/sg-evals/envoy--v1.31.2/source/common/stats/tag_producer_impl.h",
5+
"github.com/sg-evals/envoy--v1.31.2/envoy/stats/store.h",
6+
"github.com/sg-evals/envoy--v1.31.2/source/common/stats/tag_extractor_impl.h",
7+
"github.com/sg-evals/envoy--v1.31.2/envoy/stats/tag_producer.h",
8+
"github.com/sg-evals/envoy--v1.31.2/envoy/stats/tag_extractor.h",
9+
"github.com/sg-evals/envoy--v1.31.2/source/docs/stats.md"
10+
],
11+
"symbols": [],
12+
"repo_set_id": "envoy-service-mesh",
413
"_metadata": {
5-
"oracle_type": "tbd",
6-
"status": "pending_curation"
14+
"discovery_method": "sourcegraph_mcp_keyword_search",
15+
"timestamp": "2026-02-28T00:00:00Z",
16+
"repos_searched": [
17+
"sg-evals/envoy--v1.31.2",
18+
"sg-evals/grpc--957dba5e"
19+
]
720
}
8-
}
21+
}

‎benchmarks/ccb_mcp_org/ccx-agentic-228/tests/oracle_answer.json‎

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,17 @@
11
{
2-
"files": [],
3-
"text": "Oracle not yet curated \u2014 task stub.",
2+
"files": [
3+
"github.com/sg-evals/envoy--v1.31.2/source/common/listener_manager/listener_manager_impl.h",
4+
"github.com/sg-evals/envoy--v1.31.2/source/common/listener_manager/listener_manager_impl.cc",
5+
"github.com/sg-evals/envoy--v1.31.2/source/server/listener_manager_factory.h"
6+
],
7+
"symbols": [],
8+
"repo_set_id": "envoy-service-mesh",
49
"_metadata": {
5-
"oracle_type": "tbd",
6-
"status": "pending_curation"
10+
"discovery_method": "sourcegraph_mcp_keyword_search",
11+
"timestamp": "2026-02-28T00:00:00Z",
12+
"repos_searched": [
13+
"sg-evals/envoy--v1.31.2",
14+
"sg-evals/grpc--957dba5e"
15+
]
716
}
8-
}
17+
}

‎benchmarks/ccb_mcp_org/ccx-agentic-229/tests/oracle_answer.json‎

Lines changed: 15 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,18 @@
11
{
2-
"files": [],
3-
"text": "Oracle not yet curated \u2014 task stub.",
2+
"files": [
3+
"github.com/sg-evals/grafana--26d36ec/pkg/services/ngalert/ngalert.go",
4+
"github.com/sg-evals/grafana--26d36ec/pkg/services/ngalert/notifier/alertmanager.go",
5+
"github.com/sg-evals/grafana--26d36ec/pkg/services/ngalert/notifier/multiorg_alertmanager.go",
6+
"github.com/sg-evals/grafana--26d36ec/pkg/services/ngalert/store/database.go",
7+
"github.com/sg-evals/grafana--26d36ec/pkg/services/ngalert/api/api.go"
8+
],
9+
"symbols": [],
10+
"repo_set_id": "grafana-observability",
411
"_metadata": {
5-
"oracle_type": "tbd",
6-
"status": "pending_curation"
12+
"discovery_method": "sourcegraph_mcp_keyword_search",
13+
"timestamp": "2026-02-28T00:00:00Z",
14+
"repos_searched": [
15+
"sg-evals/grafana--26d36ec"
16+
]
717
}
8-
}
18+
}

‎benchmarks/ccb_mcp_org/ccx-agentic-230/tests/oracle_answer.json‎

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,16 @@
11
{
2-
"files": [],
3-
"text": "Oracle not yet curated \u2014 task stub.",
2+
"files": [
3+
"github.com/sg-evals/grafana--26d36ec/packages/grafana-prometheus/src/querycache/QueryCache.ts",
4+
"github.com/sg-evals/grafana--26d36ec/public/app/features/query/state/queryAnalytics.ts",
5+
"github.com/sg-evals/grafana--26d36ec/packages/grafana-runtime/src/utils/DataSourceWithBackend.ts"
6+
],
7+
"symbols": [],
8+
"repo_set_id": "grafana-observability",
49
"_metadata": {
5-
"oracle_type": "tbd",
6-
"status": "pending_curation"
10+
"discovery_method": "sourcegraph_mcp_keyword_search",
11+
"timestamp": "2026-02-28T00:00:00Z",
12+
"repos_searched": [
13+
"sg-evals/grafana--26d36ec"
14+
]
715
}
8-
}
16+
}

‎benchmarks/ccb_mcp_org/ccx-agentic-231/tests/oracle_answer.json‎

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,17 @@
11
{
2-
"files": [],
3-
"text": "Oracle not yet curated \u2014 task stub.",
2+
"files": [
3+
"github.com/sg-evals/grafana--26d36ec/pkg/services/org/orgimpl/store.go",
4+
"github.com/sg-evals/grafana--26d36ec/pkg/services/org/model.go",
5+
"github.com/sg-evals/grafana--26d36ec/pkg/api/org_users.go",
6+
"github.com/sg-evals/grafana--26d36ec/pkg/services/sqlstore/migrations/org_mig.go"
7+
],
8+
"symbols": [],
9+
"repo_set_id": "grafana-observability",
410
"_metadata": {
5-
"oracle_type": "tbd",
6-
"status": "pending_curation"
11+
"discovery_method": "sourcegraph_mcp_keyword_search",
12+
"timestamp": "2026-02-28T00:00:00Z",
13+
"repos_searched": [
14+
"sg-evals/grafana--26d36ec"
15+
]
716
}
8-
}
17+
}

‎benchmarks/ccb_mcp_org/ccx-agentic-232/tests/oracle_answer.json‎

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,21 @@
11
{
2-
"files": [],
3-
"text": "Oracle not yet curated \u2014 task stub.",
2+
"files": [
3+
"github.com/sg-evals/kafka--0753c489/core/src/main/scala/kafka/server/ReplicaFetcherThread.scala",
4+
"github.com/sg-evals/kafka--0753c489/core/src/main/scala/kafka/server/ReplicaFetcherManager.scala",
5+
"github.com/sg-evals/kafka--0753c489/storage/src/main/java/org/apache/kafka/storage/internals/epoch/LeaderEpochFileCache.java",
6+
"github.com/sg-evals/kafka--0753c489/storage/src/main/java/org/apache/kafka/storage/internals/log/LogLoader.java",
7+
"github.com/sg-evals/kafka--0753c489/core/src/main/scala/kafka/server/ReplicaAlterLogDirsThread.scala",
8+
"github.com/sg-evals/kafka--0753c489/storage/src/main/java/org/apache/kafka/server/log/remote/storage/RemoteLogManager.java"
9+
],
10+
"symbols": [],
11+
"repo_set_id": "apache-kafka-ecosystem",
412
"_metadata": {
5-
"oracle_type": "tbd",
6-
"status": "pending_curation"
13+
"discovery_method": "sourcegraph_mcp_keyword_search",
14+
"timestamp": "2026-02-28T00:00:00Z",
15+
"repos_searched": [
16+
"sg-evals/kafka--0753c489",
17+
"sg-evals/flink--0cc95fcc",
18+
"sg-evals/camel--1006f047"
19+
]
720
}
8-
}
21+
}

0 commit comments

Comments
 (0)