Skip to content

Conversation

@cfsmp3
Copy link
Contributor

@cfsmp3 cfsmp3 commented Dec 25, 2025

Summary

  • Fixes GitHub status message not matching the actual state for PR tests
  • When comment_pr() returns a different state than the initial crash/results check, the message is now updated to match

Problem

When a PR's tests complete, the code:

  1. Sets initial state/message based on crashes and result mismatches
  2. For PRs, calls comment_pr() which returns a potentially different state (SUCCESS if no new failures compared to master)
  3. The state gets updated, but the message was never updated to match

This caused confusing situations like test 7203 where:

  • All 260 tests passed on the sample platform page
  • GitHub showed a green checkmark (success state)
  • But the message said "Not all tests completed successfully, please check"

Root Cause

if crashes > 0 or results > 0:
    state = Status.FAILURE
    message = 'Not all tests completed successfully, please check'
else:
    state = Status.SUCCESS
    message = 'Tests completed'

if test.test_type == TestType.pull_request:
    state = comment_pr(test)  # ← State overwritten, message NOT updated!

The comment_pr() function uses different logic - it returns SUCCESS if there are no new failures compared to master (pre-existing failures are OK). The initial check was finding some hash mismatches, but those were pre-existing on master.

Solution

Update the message after comment_pr() returns to match the new state:

  • SUCCESS → "All tests passed"
  • FAILURE → "Not all tests completed successfully, please check"

Test plan

  • Existing tests pass (test_comments_successfully_in_passed_pr_test, test_comments_successfuly_in_failed_pr_test)
  • Verify next PR test shows correct message on GitHub

🤖 Generated with Claude Code

When a PR's tests complete, the code sets the initial state and message
based on crashes/results. Then for PRs, comment_pr() returns a potentially
different state (SUCCESS if no NEW failures vs master). However, the message
was never updated to match this new state.

This caused confusing situations where all tests passed (state=SUCCESS)
but the message said "Not all tests completed successfully".

Now the message is updated after comment_pr() to match the returned state:
- SUCCESS: "All tests passed"
- FAILURE: "Not all tests completed successfully, please check"

Fixes issue where test 7203 showed all tests passed on the sample platform
but GitHub showed "Not all tests completed successfully".

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@sonarqubecloud
Copy link

@cfsmp3 cfsmp3 merged commit ee6b224 into master Dec 25, 2025
6 checks passed
@cfsmp3 cfsmp3 deleted the fix/pr-status-message-mismatch branch December 25, 2025 15:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants