fix(ci): preflight-main race mit qodana-run synchronisieren#102
fix(ci): preflight-main race mit qodana-run synchronisieren#102tomtastisch merged 7 commits intomainfrom
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bb280c9654
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Pull request overview
Diese PR synchronisiert den code-scanning-tools-zero-Preflight-Check auf main, indem vor der Abfrage offener Code-Scanning-Alerts auf den Abschluss des qodana-Workflow-Runs (SARIF-Upload) gewartet wird, um Race-bedingte False-Fails zu vermeiden.
Changes:
- Warteschleife fuer
qodana-Workflow-Run (gleicherGITHUB_SHA) vor Alert-Query aufmain. - Fail-closed Verhalten bei API-/Statusfehlern bzw. nicht-
successConclusions mit aussagekraeftigem Logging. - Danach unveraendert: Query der offenen Code-Scanning-Alerts und Fail bei
count > 0.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 7 comments.
Comments suppressed due to low confidence (2)
tools/ci/check-code-scanning-tools-zero.sh:208
- Incorrect indentation: the code inside the if block (lines 121-208) should be indented further than the if statement itself (line 120).
Currently, wait_attempt=1 and subsequent lines are at the same indentation level as the if statement, which violates standard shell scripting conventions and makes the code harder to read. All lines from 121 to 208 (up to the done statement) should be indented by at least 2 additional spaces relative to line 120.
wait_attempt=1
# Qodana can be queued behind runner load; allow a generous fail-closed window.
wait_max_attempts=120
wait_delay=10
run_event_filter="${EVENT_NAME:-push}"
pr_head_sha=""
pr_head_ref=""
if [[ "${EVENT_NAME}" == "pull_request" && -n "${GITHUB_EVENT_PATH:-}" && -f "${GITHUB_EVENT_PATH:-}" ]]; then
pr_head_sha="$(jq -r '.pull_request.head.sha // empty' "${GITHUB_EVENT_PATH}")"
pr_head_ref="$(jq -r '.pull_request.head.ref // empty' "${GITHUB_EVENT_PATH}")"
fi
while true; do
qodana_runs_json="${OUT_DIR}/qodana-runs.json"
api_path="repos/${REPO}/actions/runs?event=${run_event_filter}&per_page=100"
if [[ "${EVENT_NAME}" != "pull_request" ]]; then
api_path="${api_path}&head_sha=${GITHUB_SHA}"
fi
if ! gh api "${api_path}" > "${qodana_runs_json}" 2>> "${RAW_LOG}"; then
if (( wait_attempt >= wait_max_attempts )); then
fail "Qodana-Runstatus fuer aktuellen SHA konnte nicht geladen werden"
fi
log "WARN: Qodana-Runstatus API-Fehler, retry ${wait_attempt}/${wait_max_attempts}"
sleep "${wait_delay}"
wait_attempt=$((wait_attempt + 1))
continue
fi
if [[ "${EVENT_NAME}" == "pull_request" ]]; then
qodana_fields="$(jq -r \
--arg pr_head_sha "${pr_head_sha}" \
--arg pr_head_ref "${pr_head_ref}" \
'.workflow_runs
| map(select(.name=="qodana"))
| map(select((($pr_head_sha != "") and (.head_sha == $pr_head_sha)) or (($pr_head_ref != "") and (.head_branch == $pr_head_ref))))
| sort_by(.created_at)
| reverse
| .[0]
| [(.status // ""), (.conclusion // ""), (.html_url // "")]
| @tsv' "${qodana_runs_json}")"
else
qodana_fields="$(jq -r \
'.workflow_runs
| map(select(.name=="qodana"))
| sort_by(.created_at)
| reverse
| .[0]
| [(.status // ""), (.conclusion // ""), (.html_url // "")]
| @tsv' "${qodana_runs_json}")"
fi
IFS=$'\t' read -r qodana_status qodana_conclusion qodana_url <<< "${qodana_fields:-}"
if [[ -z "${qodana_status}" ]]; then
if (( wait_attempt >= wait_max_attempts )); then
fail "Kein qodana-Run fuer aktuellen Kontext gefunden (sha=${GITHUB_SHA}, pr_head_sha=${pr_head_sha:-n/a}, pr_head_ref=${pr_head_ref:-n/a})"
fi
log "INFO: qodana-Run noch nicht sichtbar (retry ${wait_attempt}/${wait_max_attempts})"
sleep "${wait_delay}"
wait_attempt=$((wait_attempt + 1))
continue
fi
if [[ "${qodana_status}" != "completed" ]]; then
if (( wait_attempt >= wait_max_attempts )); then
fail "qodana-Run fuer SHA=${GITHUB_SHA} ist nicht abgeschlossen (status=${qodana_status})"
fi
log "INFO: warte auf qodana-Runabschluss (status=${qodana_status}, retry ${wait_attempt}/${wait_max_attempts})"
sleep "${wait_delay}"
wait_attempt=$((wait_attempt + 1))
continue
fi
if [[ "${qodana_conclusion}" != "success" ]]; then
fail "qodana-Run fuer SHA=${GITHUB_SHA} ist fehlgeschlagen (conclusion=${qodana_conclusion:-unknown}, url=${qodana_url:-n/a})"
fi
log "INFO: qodana-Run fuer SHA=${GITHUB_SHA} erfolgreich abgeschlossen (${qodana_url:-n/a})"
break
done
fi
fi
attempt=1
delay=2
max_attempts=3
while true; do
api_path="repos/${REPO}/code-scanning/alerts?state=open&per_page=100"
if [[ -n "${QUERY_REF}" ]]; then
api_path="${api_path}&ref=${QUERY_REF}"
tools/ci/check-code-scanning-tools-zero.sh:210
- The wait logic doesn't account for scenarios where the qodana workflow is skipped by its job-level condition.
According to .github/workflows/qodana.yml line 16, the qodana job is skipped for:
- Fork PRs (github.event.pull_request.head.repo.fork == true)
- Dependabot PRs (github.actor == 'dependabot[bot]')
In these cases, this script will wait up to 20 minutes for a qodana run that will never appear, then fail with "Kein qodana-Run fuer aktuellen Kontext gefunden".
Consider:
- Detecting these skip conditions before entering the wait loop
- Skipping the qodana wait for fork/dependabot PRs
- Or documenting that such PRs are expected to fail this check (if that's the intended behavior)
if [[ "${should_wait_for_qodana}" == "true" ]]; then
wait_attempt=1
# Qodana can be queued behind runner load; allow a generous fail-closed window.
wait_max_attempts=120
wait_delay=10
run_event_filter="${EVENT_NAME:-push}"
pr_head_sha=""
pr_head_ref=""
if [[ "${EVENT_NAME}" == "pull_request" && -n "${GITHUB_EVENT_PATH:-}" && -f "${GITHUB_EVENT_PATH:-}" ]]; then
pr_head_sha="$(jq -r '.pull_request.head.sha // empty' "${GITHUB_EVENT_PATH}")"
pr_head_ref="$(jq -r '.pull_request.head.ref // empty' "${GITHUB_EVENT_PATH}")"
fi
while true; do
qodana_runs_json="${OUT_DIR}/qodana-runs.json"
api_path="repos/${REPO}/actions/runs?event=${run_event_filter}&per_page=100"
if [[ "${EVENT_NAME}" != "pull_request" ]]; then
api_path="${api_path}&head_sha=${GITHUB_SHA}"
fi
if ! gh api "${api_path}" > "${qodana_runs_json}" 2>> "${RAW_LOG}"; then
if (( wait_attempt >= wait_max_attempts )); then
fail "Qodana-Runstatus fuer aktuellen SHA konnte nicht geladen werden"
fi
log "WARN: Qodana-Runstatus API-Fehler, retry ${wait_attempt}/${wait_max_attempts}"
sleep "${wait_delay}"
wait_attempt=$((wait_attempt + 1))
continue
fi
if [[ "${EVENT_NAME}" == "pull_request" ]]; then
qodana_fields="$(jq -r \
--arg pr_head_sha "${pr_head_sha}" \
--arg pr_head_ref "${pr_head_ref}" \
'.workflow_runs
| map(select(.name=="qodana"))
| map(select((($pr_head_sha != "") and (.head_sha == $pr_head_sha)) or (($pr_head_ref != "") and (.head_branch == $pr_head_ref))))
| sort_by(.created_at)
| reverse
| .[0]
| [(.status // ""), (.conclusion // ""), (.html_url // "")]
| @tsv' "${qodana_runs_json}")"
else
qodana_fields="$(jq -r \
'.workflow_runs
| map(select(.name=="qodana"))
| sort_by(.created_at)
| reverse
| .[0]
| [(.status // ""), (.conclusion // ""), (.html_url // "")]
| @tsv' "${qodana_runs_json}")"
fi
IFS=$'\t' read -r qodana_status qodana_conclusion qodana_url <<< "${qodana_fields:-}"
if [[ -z "${qodana_status}" ]]; then
if (( wait_attempt >= wait_max_attempts )); then
fail "Kein qodana-Run fuer aktuellen Kontext gefunden (sha=${GITHUB_SHA}, pr_head_sha=${pr_head_sha:-n/a}, pr_head_ref=${pr_head_ref:-n/a})"
fi
log "INFO: qodana-Run noch nicht sichtbar (retry ${wait_attempt}/${wait_max_attempts})"
sleep "${wait_delay}"
wait_attempt=$((wait_attempt + 1))
continue
fi
if [[ "${qodana_status}" != "completed" ]]; then
if (( wait_attempt >= wait_max_attempts )); then
fail "qodana-Run fuer SHA=${GITHUB_SHA} ist nicht abgeschlossen (status=${qodana_status})"
fi
log "INFO: warte auf qodana-Runabschluss (status=${qodana_status}, retry ${wait_attempt}/${wait_max_attempts})"
sleep "${wait_delay}"
wait_attempt=$((wait_attempt + 1))
continue
fi
if [[ "${qodana_conclusion}" != "success" ]]; then
fail "qodana-Run fuer SHA=${GITHUB_SHA} ist fehlgeschlagen (conclusion=${qodana_conclusion:-unknown}, url=${qodana_url:-n/a})"
fi
log "INFO: qodana-Run fuer SHA=${GITHUB_SHA} erfolgreich abgeschlossen (${qodana_url:-n/a})"
break
done
fi
fi
attempt=1
delay=2
max_attempts=3
while true; do
api_path="repos/${REPO}/code-scanning/alerts?state=open&per_page=100"
if [[ -n "${QUERY_REF}" ]]; then
api_path="${api_path}&ref=${QUERY_REF}"
fi
if gh api "${api_path}" --paginate > "${ALERTS_JSON}" 2>> "${RAW_LOG}"; then
Ziel & Scope
preflightundqodanaaufmainbeseitigen, damitcode-scanning-tools-zeronicht vor SARIF-Upload false-negative/fail liefert.tools/ci/check-code-scanning-tools-zero.sh.Umgesetzte Aufgaben (abhaken)
pushnachmainwartet der Check jetzt deterministisch auf denqodana-Run fuer denselbenGITHUB_SHA.Nachbesserungen aus Review (iterativ)
Security- und Merge-Gates
preflightfail.mainwerden vermieden.security/code-scanning/toolsund0 offene Alertsfuer policy-relevante Findings.Evidence (auditierbar)
GITHUB_EVENT_NAME=push GITHUB_REF=refs/heads/main GITHUB_SHA=29ad98731bcd941b7ec9709bccaae4ed3b4993ab GITHUB_REPOSITORY=tomtastisch/FileClassifier ./tools/ci/check-code-scanning-tools-zero.shjq -r '.status+" | "+(.open_alerts|tostring)' artifacts/ci/preflight/code-scanning-tools-zero/result.json=>pass | 0gh run download 22186363050 -n ci-preflight ...mitOffene Code-Scanning-Alerts vorhanden (66).DoD (mindestens 2 pro Punkt)
Punkt A: Race-Synchronisierung
mainden qodana-Run anhandhead_sha.Punkt B: Fail-closed Robustheit
fail.Timeoutfenster fuer qodana-Synchronisierung auf 20 Minuten erweitert (Queue-/Lastspitzen fail-closed abgedeckt).
Label-Mapping erweitert: Aenderungen an
tools/ci/check-code-scanning-tools-zero.shsetzenarea:qodana.