Skip to content

Commit 272c73a

Browse files
sjarmakclaude
andcommitted
Add timeout guards to SWE-bench Pro test.sh verifiers
Jest/npm/pytest/go test commands in 5 task verifiers had no timeout, causing indefinite hangs when individual tests stall (observed: InteractiveAuthDialog-test.ts hung >2h in element-web task). Adds `timeout 600` (10min) wrapper to all affected test runners. Also adds `--forceExit` to jest commands to kill workers immediately when tests complete (prevents jest from waiting for open handles). Affected tasks: - element-web-roomheaderbuttons-can-crash-fix-001 (jest) - element-web-unread-indicators-diverge-fix-001 (jest) - vscode-stale-diagnostics-feat-001 (npm test) - ansible-abc-imports-fix-001 (pytest) - teleport-users-can-delete-fix-001 (go test) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 3c487d0 commit 272c73a

File tree

5 files changed

+5
-5
lines changed
  • benchmarks
    • csb_sdlc_feature/vscode-stale-diagnostics-feat-001/tests
    • csb_sdlc_fix
      • ansible-abc-imports-fix-001/tests
      • element-web-roomheaderbuttons-can-crash-fix-001/tests
      • element-web-unread-indicators-diverge-fix-001/tests
      • teleport-users-can-delete-fix-001/tests

5 files changed

+5
-5
lines changed

benchmarks/csb_sdlc_feature/vscode-stale-diagnostics-feat-001/tests/test.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -262,7 +262,7 @@ UNIT_TEST_PASS=1
262262
if [ -f "extensions/typescript-language-features/package.json" ]; then
263263
# VS Code extensions use npm test or a custom test runner
264264
cd extensions/typescript-language-features
265-
npm test 2>/logs/verifier/test_errors.txt && TEST_RC=0 || TEST_RC=$?
265+
timeout 600 npm test 2>/logs/verifier/test_errors.txt && TEST_RC=0 || TEST_RC=$?
266266
cd "$TASK_REPO_ROOT"
267267
if [ "$TEST_RC" -eq 0 ]; then
268268
echo "[x] TypeScript language features tests passed"

benchmarks/csb_sdlc_fix/ansible-abc-imports-fix-001/tests/test.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -182,7 +182,7 @@ echo "Test modules: test_recursive_finder.py (TestRecursiveFinder) + test_collec
182182
# Individual test IDs are NOT used because parametrized tests contain Czech and
183183
# Ukrainian characters whose escape sequences are not interpreted by bash,
184184
# causing pytest to find zero matching tests and score 0.0.
185-
python -m pytest \
185+
timeout 600 python -m pytest \
186186
"test/units/executor/module_common/test_recursive_finder.py::TestRecursiveFinder" \
187187
"test/units/module_utils/common/test_collections.py" \
188188
-v 2>&1 | tee test_output.log

benchmarks/csb_sdlc_fix/element-web-roomheaderbuttons-can-crash-fix-001/tests/test.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -147,7 +147,7 @@ else
147147
fi
148148

149149
echo "Running tests..."
150-
npx jest --no-cache "test/components/views/dialogs/InteractiveAuthDialog-test.ts" "test/components/views/right_panel/RoomHeaderButtons-test.tsx" "test/components/views/rooms/RoomHeader-test.ts" "test/components/structures/ThreadView-test.ts" "test/components/views/settings/shared/SettingsSubsection-test.ts" 2>&1 | tee test_output.log
150+
timeout 600 npx jest --no-cache --forceExit "test/components/views/dialogs/InteractiveAuthDialog-test.ts" "test/components/views/right_panel/RoomHeaderButtons-test.tsx" "test/components/views/rooms/RoomHeader-test.ts" "test/components/structures/ThreadView-test.ts" "test/components/views/settings/shared/SettingsSubsection-test.ts" 2>&1 | tee test_output.log
151151
TEST_EXIT_CODE=$?
152152

153153
# Write reward for Harbor with partial credit

benchmarks/csb_sdlc_fix/element-web-unread-indicators-diverge-fix-001/tests/test.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -506,7 +506,7 @@ else
506506
fi
507507

508508
echo "Running tests..."
509-
npx jest --no-cache "test/test-utils/threads.ts" "test/editor/position-test.ts" "test/components/views/typography/Caption-test.ts" "test/components/structures/UserMenu-test.ts" "test/components/views/settings/devices/SelectableDeviceTile-test.ts" 2>&1 | tee test_output.log
509+
timeout 600 npx jest --no-cache --forceExit "test/test-utils/threads.ts" "test/editor/position-test.ts" "test/components/views/typography/Caption-test.ts" "test/components/structures/UserMenu-test.ts" "test/components/views/settings/devices/SelectableDeviceTile-test.ts" 2>&1 | tee test_output.log
510510
TEST_EXIT_CODE=$?
511511

512512
# Write reward for Harbor with partial credit

benchmarks/csb_sdlc_fix/teleport-users-can-delete-fix-001/tests/test.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -220,7 +220,7 @@ else
220220
fi
221221

222222
echo "Running tests..."
223-
go test -v 2>&1 | tee test_output.log
223+
timeout 600 go test -v 2>&1 | tee test_output.log
224224
TEST_EXIT_CODE=$?
225225

226226
# Write reward for Harbor with partial credit

0 commit comments

Comments
 (0)