Skip to content

Commit a31d90c

Browse files
sjarmakclaude
sjarmak
and
claude
committed
feat: add Dockerfile.artifact_only for all 12 MCP-unique tasks
All ccb_mcp_* tasks are write-only (verifier scores answer.json via oracle_checks.py). Each Dockerfile.artifact_only clones the same repos as the original Dockerfile and adds the /tmp/.artifact_only_mode sentinel. No /repo_full backup needed since verification has no source-code dependency. Covers 6 suites: crossorg (2), crossrepo_tracing (3), incident (1), onboarding (3), platform (1), security (2). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 8a83f02 commit a31d90c

File tree

12 files changed

+467
-0
lines changed

12 files changed

+467
-0
lines changed

‎benchmarks/ccb_mcp_crossorg/ccx-crossorg-061/environment/Dockerfile.artifact_only‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# ccx-crossorg-061 — artifact_only variant
2+
# Repos cloned for baseline agent to read locally.
3+
# MCP agent deletes source files at runtime via agent startup script.
4+
# Verifier (oracle_checks.py) scores answer.json only — no repo restore needed.
5+
6+
FROM ubuntu:22.04
7+
8+
ENV DEBIAN_FRONTEND=noninteractive
9+
10+
# Base tools
11+
RUN apt-get update && apt-get install -y --no-install-recommends \
12+
git \
13+
ca-certificates \
14+
curl \
15+
python3 \
16+
golang-go \
17+
&& rm -rf /var/lib/apt/lists/*
18+
19+
WORKDIR /workspace
20+
21+
# Clone local checkout repos (baseline config: agent has local access to these)
22+
RUN git clone --depth 1 --branch v1.32.0 https://github.com/kubernetes/kubernetes /workspace/kubernetes
23+
RUN git clone --depth 1 --branch v11.4.0 https://github.com/grafana/grafana /workspace/grafana
24+
RUN git clone --depth 1 --branch v3.5.17 https://github.com/etcd-io/etcd /workspace/etcd
25+
26+
# Initialize git identity for agent commits
27+
RUN git config --global user.email "agent@example.com" && \
28+
git config --global user.name "Agent" && \
29+
git config --global safe.directory '*'
30+
31+
# Create log directories
32+
RUN mkdir -p /logs/agent /logs/verifier
33+
34+
# --- artifact_only mode ---
35+
# Sentinel flag for artifact-based verification.
36+
# Source stays readable for baseline agent; MCP agent deletes at runtime.
37+
RUN touch /tmp/.artifact_only_mode
38+
39+
ENTRYPOINT []

‎benchmarks/ccb_mcp_crossorg/ccx-crossorg-066/environment/Dockerfile.artifact_only‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# ccx-crossorg-066 — artifact_only variant
2+
# Repos cloned for baseline agent to read locally.
3+
# MCP agent deletes source files at runtime via agent startup script.
4+
# Verifier (oracle_checks.py) scores answer.json only — no repo restore needed.
5+
6+
FROM ubuntu:22.04
7+
8+
ENV DEBIAN_FRONTEND=noninteractive
9+
10+
# Base tools
11+
RUN apt-get update && apt-get install -y --no-install-recommends \
12+
git \
13+
ca-certificates \
14+
curl \
15+
python3 \
16+
golang-go \
17+
&& rm -rf /var/lib/apt/lists/*
18+
19+
WORKDIR /workspace
20+
21+
# Clone local checkout repos (baseline config: agent has local access to these)
22+
RUN git clone --depth 1 --branch v1.32.0 https://github.com/kubernetes/kubernetes /workspace/kubernetes
23+
RUN git clone --depth 1 --branch v11.4.0 https://github.com/grafana/grafana /workspace/grafana
24+
RUN git clone --depth 1 --branch v3.5.17 https://github.com/etcd-io/etcd /workspace/etcd
25+
26+
# Initialize git identity for agent commits
27+
RUN git config --global user.email "agent@example.com" && \
28+
git config --global user.name "Agent" && \
29+
git config --global safe.directory '*'
30+
31+
# Create log directories
32+
RUN mkdir -p /logs/agent /logs/verifier
33+
34+
# --- artifact_only mode ---
35+
# Sentinel flag for artifact-based verification.
36+
# Source stays readable for baseline agent; MCP agent deletes at runtime.
37+
RUN touch /tmp/.artifact_only_mode
38+
39+
ENTRYPOINT []

‎benchmarks/ccb_mcp_crossrepo_tracing/ccx-config-trace-010/environment/Dockerfile.artifact_only‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# ccx-config-trace-010 — artifact_only variant
2+
# Repos cloned for baseline agent to read locally.
3+
# MCP agent deletes source files at runtime via agent startup script.
4+
# Verifier (oracle_checks.py) scores answer.json only — no repo restore needed.
5+
6+
FROM ubuntu:22.04
7+
8+
ENV DEBIAN_FRONTEND=noninteractive
9+
10+
# Base tools
11+
RUN apt-get update && apt-get install -y --no-install-recommends \
12+
git \
13+
ca-certificates \
14+
curl \
15+
python3 \
16+
golang-go \
17+
&& rm -rf /var/lib/apt/lists/*
18+
19+
WORKDIR /workspace
20+
21+
# Clone local checkout repos (baseline config: agent has local access to these)
22+
RUN git clone --depth 1 --branch v1.32.0 https://github.com/kubernetes/kubernetes /workspace/kubernetes
23+
RUN git clone https://github.com/kubernetes/client-go /workspace/client-go --no-checkout && \
24+
cd /workspace/client-go && git fetch --depth 1 origin 8020fc4fcf89965904a5f43689f169d6e01d1e80 && git checkout FETCH_HEAD
25+
26+
# Initialize git identity for agent commits
27+
RUN git config --global user.email "agent@example.com" && \
28+
git config --global user.name "Agent" && \
29+
git config --global safe.directory '*'
30+
31+
# Create log directories
32+
RUN mkdir -p /logs/agent /logs/verifier
33+
34+
# --- artifact_only mode ---
35+
# Sentinel flag for artifact-based verification.
36+
# Source stays readable for baseline agent; MCP agent deletes at runtime.
37+
RUN touch /tmp/.artifact_only_mode
38+
39+
ENTRYPOINT []

‎benchmarks/ccb_mcp_crossrepo_tracing/ccx-dep-trace-001/environment/Dockerfile.artifact_only‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# ccx-dep-trace-001 — artifact_only variant
2+
# Repos cloned for baseline agent to read locally.
3+
# MCP agent deletes source files at runtime via agent startup script.
4+
# Verifier (oracle_checks.py) scores answer.json only — no repo restore needed.
5+
6+
FROM ubuntu:22.04
7+
8+
ENV DEBIAN_FRONTEND=noninteractive
9+
10+
# Base tools
11+
RUN apt-get update && apt-get install -y --no-install-recommends \
12+
git \
13+
ca-certificates \
14+
curl \
15+
python3 \
16+
golang-go \
17+
&& rm -rf /var/lib/apt/lists/*
18+
19+
WORKDIR /workspace
20+
21+
# Clone local checkout repos (baseline config: agent has local access to these)
22+
RUN git clone --depth 1 --branch v1.32.0 https://github.com/kubernetes/kubernetes /workspace/kubernetes
23+
RUN git clone https://github.com/kubernetes/client-go /workspace/client-go --no-checkout && \
24+
cd /workspace/client-go && git fetch --depth 1 origin 8020fc4fcf89965904a5f43689f169d6e01d1e80 && git checkout FETCH_HEAD
25+
26+
# Initialize git identity for agent commits
27+
RUN git config --global user.email "agent@example.com" && \
28+
git config --global user.name "Agent" && \
29+
git config --global safe.directory '*'
30+
31+
# Create log directories
32+
RUN mkdir -p /logs/agent /logs/verifier
33+
34+
# --- artifact_only mode ---
35+
# Sentinel flag for artifact-based verification.
36+
# Source stays readable for baseline agent; MCP agent deletes at runtime.
37+
RUN touch /tmp/.artifact_only_mode
38+
39+
ENTRYPOINT []

‎benchmarks/ccb_mcp_crossrepo_tracing/ccx-dep-trace-004/environment/Dockerfile.artifact_only‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# ccx-dep-trace-004 — artifact_only variant
2+
# Repos cloned for baseline agent to read locally.
3+
# MCP agent deletes source files at runtime via agent startup script.
4+
# Verifier (oracle_checks.py) scores answer.json only — no repo restore needed.
5+
6+
FROM ubuntu:22.04
7+
8+
ENV DEBIAN_FRONTEND=noninteractive
9+
10+
# Base tools
11+
RUN apt-get update && apt-get install -y --no-install-recommends \
12+
git \
13+
ca-certificates \
14+
curl \
15+
python3 \
16+
golang-go \
17+
&& rm -rf /var/lib/apt/lists/*
18+
19+
WORKDIR /workspace
20+
21+
# Clone local checkout repos (baseline config: agent has local access to these)
22+
RUN git clone --depth 1 --branch v11.4.0 https://github.com/grafana/grafana /workspace/grafana
23+
RUN git clone https://github.com/grafana/loki /workspace/loki --no-checkout && \
24+
cd /workspace/loki && git fetch --depth 1 origin a3af38d4da899032d3ee46a30932d072d37e1b9c && git checkout FETCH_HEAD
25+
26+
# Initialize git identity for agent commits
27+
RUN git config --global user.email "agent@example.com" && \
28+
git config --global user.name "Agent" && \
29+
git config --global safe.directory '*'
30+
31+
# Create log directories
32+
RUN mkdir -p /logs/agent /logs/verifier
33+
34+
# --- artifact_only mode ---
35+
# Sentinel flag for artifact-based verification.
36+
# Source stays readable for baseline agent; MCP agent deletes at runtime.
37+
RUN touch /tmp/.artifact_only_mode
38+
39+
ENTRYPOINT []

‎benchmarks/ccb_mcp_incident/ccx-incident-031/environment/Dockerfile.artifact_only‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# ccx-incident-031 — artifact_only variant
2+
# Repos cloned for baseline agent to read locally.
3+
# MCP agent deletes source files at runtime via agent startup script.
4+
# Verifier (oracle_checks.py) scores answer.json only — no repo restore needed.
5+
6+
FROM ubuntu:22.04
7+
8+
ENV DEBIAN_FRONTEND=noninteractive
9+
10+
# Base tools
11+
RUN apt-get update && apt-get install -y --no-install-recommends \
12+
git \
13+
ca-certificates \
14+
curl \
15+
python3 \
16+
&& rm -rf /var/lib/apt/lists/*
17+
18+
WORKDIR /workspace
19+
20+
# Clone local checkout repos (baseline config: agent has local access to these)
21+
RUN git clone --depth 1 --branch v1.32.0 https://github.com/kubernetes/kubernetes /workspace/kubernetes
22+
RUN git clone --depth 1 --branch v3.5.17 https://github.com/etcd-io/etcd /workspace/etcd
23+
24+
# Initialize git identity for agent commits
25+
RUN git config --global user.email "agent@example.com" && \
26+
git config --global user.name "Agent" && \
27+
git config --global safe.directory '*'
28+
29+
# Create log directories
30+
RUN mkdir -p /logs/agent /logs/verifier
31+
32+
# --- artifact_only mode ---
33+
# Sentinel flag for artifact-based verification.
34+
# Source stays readable for baseline agent; MCP agent deletes at runtime.
35+
RUN touch /tmp/.artifact_only_mode
36+
37+
ENTRYPOINT []

‎benchmarks/ccb_mcp_onboarding/ccx-explore-042-ds/environment/Dockerfile.artifact_only‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# ccx-explore-042-ds — artifact_only variant
2+
# Repos cloned for baseline agent to read locally.
3+
# MCP agent deletes source files at runtime via agent startup script.
4+
# Verifier (oracle_checks.py) scores answer.json only — no repo restore needed.
5+
6+
FROM ubuntu:22.04
7+
8+
ENV DEBIAN_FRONTEND=noninteractive
9+
10+
# Base tools
11+
RUN apt-get update && apt-get install -y --no-install-recommends \
12+
git \
13+
ca-certificates \
14+
curl \
15+
python3 \
16+
python3-pip \
17+
&& rm -rf /var/lib/apt/lists/*
18+
19+
WORKDIR /workspace
20+
21+
# Clone local checkout repos (baseline config: agent has local access to these)
22+
RUN git clone --depth 1 --branch 1.6.1 https://github.com/scikit-learn/scikit-learn /workspace/scikit-learn
23+
RUN git clone --depth 1 --branch v2.2.2 https://github.com/numpy/numpy /workspace/numpy
24+
RUN git clone --depth 1 --branch v2.2.3 https://github.com/pandas-dev/pandas /workspace/pandas
25+
RUN git clone --depth 1 --branch v1.15.1 https://github.com/scipy/scipy /workspace/scipy
26+
27+
# Initialize git identity for agent commits
28+
RUN git config --global user.email "agent@example.com" && \
29+
git config --global user.name "Agent" && \
30+
git config --global safe.directory '*'
31+
32+
# Create log directories
33+
RUN mkdir -p /logs/agent /logs/verifier
34+
35+
# --- artifact_only mode ---
36+
# Sentinel flag for artifact-based verification.
37+
# Source stays readable for baseline agent; MCP agent deletes at runtime.
38+
RUN touch /tmp/.artifact_only_mode
39+
40+
ENTRYPOINT []

‎benchmarks/ccb_mcp_onboarding/ccx-onboard-041/environment/Dockerfile.artifact_only‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# ccx-onboard-041 — artifact_only variant
2+
# Repos cloned for baseline agent to read locally.
3+
# MCP agent deletes source files at runtime via agent startup script.
4+
# Verifier (oracle_checks.py) scores answer.json only — no repo restore needed.
5+
6+
FROM ubuntu:22.04
7+
8+
ENV DEBIAN_FRONTEND=noninteractive
9+
10+
# Base tools
11+
RUN apt-get update && apt-get install -y --no-install-recommends \
12+
git \
13+
ca-certificates \
14+
curl \
15+
python3 \
16+
python3-pip \
17+
&& rm -rf /var/lib/apt/lists/*
18+
19+
WORKDIR /workspace
20+
21+
# Clone local checkout repos (baseline config: agent has local access to these)
22+
RUN git clone --depth 1 --branch 1.6.1 https://github.com/scikit-learn/scikit-learn /workspace/scikit-learn
23+
RUN git clone --depth 1 --branch v2.2.3 https://github.com/pandas-dev/pandas /workspace/pandas
24+
25+
# Initialize git identity for agent commits
26+
RUN git config --global user.email "agent@example.com" && \
27+
git config --global user.name "Agent" && \
28+
git config --global safe.directory '*'
29+
30+
# Create log directories
31+
RUN mkdir -p /logs/agent /logs/verifier
32+
33+
# --- artifact_only mode ---
34+
# Sentinel flag for artifact-based verification.
35+
# Source stays readable for baseline agent; MCP agent deletes at runtime.
36+
RUN touch /tmp/.artifact_only_mode
37+
38+
ENTRYPOINT []

‎benchmarks/ccb_mcp_onboarding/ccx-onboard-050-ds/environment/Dockerfile.artifact_only‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# ccx-onboard-050-ds — artifact_only variant
2+
# Repos cloned for baseline agent to read locally.
3+
# MCP agent deletes source files at runtime via agent startup script.
4+
# Verifier (oracle_checks.py) scores answer.json only — no repo restore needed.
5+
6+
FROM ubuntu:22.04
7+
8+
ENV DEBIAN_FRONTEND=noninteractive
9+
10+
# Base tools
11+
RUN apt-get update && apt-get install -y --no-install-recommends \
12+
git \
13+
ca-certificates \
14+
curl \
15+
python3 \
16+
golang-go \
17+
&& rm -rf /var/lib/apt/lists/*
18+
19+
WORKDIR /workspace
20+
21+
# Clone local checkout repos (baseline config: agent has local access to these)
22+
RUN git clone --depth 1 --branch v1.32.0 https://github.com/kubernetes/kubernetes /workspace/kubernetes
23+
RUN git clone --depth 1 --branch v3.5.17 https://github.com/etcd-io/etcd /workspace/etcd
24+
RUN git clone https://github.com/kubernetes/client-go /workspace/client-go --no-checkout && \
25+
cd /workspace/client-go && git fetch --depth 1 origin 8020fc4fcf89965904a5f43689f169d6e01d1e80 && git checkout FETCH_HEAD
26+
27+
# Initialize git identity for agent commits
28+
RUN git config --global user.email "agent@example.com" && \
29+
git config --global user.name "Agent" && \
30+
git config --global safe.directory '*'
31+
32+
# Create log directories
33+
RUN mkdir -p /logs/agent /logs/verifier
34+
35+
# --- artifact_only mode ---
36+
# Sentinel flag for artifact-based verification.
37+
# Source stays readable for baseline agent; MCP agent deletes at runtime.
38+
RUN touch /tmp/.artifact_only_mode
39+
40+
ENTRYPOINT []

0 commit comments

Comments
 (0)