-
Notifications
You must be signed in to change notification settings - Fork 49
Fix GitHub tests without a network connection #509
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
The wrapper is supposed to catch connection errors and turn them into `xfail`, but `test_rmdir_not_empty` _also_ catches connection errors because they are `OSError` (which is what it expect, thus the test still fails everything. Instead, change the `xfail_on_github_rate_limit` function to check for networking first, and mark everything `xfail` directly. Also, remove the `_xfail_on_rate_limit_errors` fixture, which is redundant to the above and doesn't work fully because it only applies to the class's direct test methods and not its inherited test methods.
|
Hi @QuLogic thank you for your contribution. I can still generate transient errors with your implementation when rerunning the checks over and over. I'll push a quick update to your PR, let me know if that solves the issue for you |
|
I'm merging this to continue with other test refactors. We can open another PR in case it's not sufficient. Thanks again 🙏 |
|
No, sorry, this is much worse. Now all tests in the file fail without a network connection instead of just |
|
Sorry to hear. Can you provide some information about your system and a full traceback? |
|
I am on the Fedora build system, which has zero network access. The original failure with DetailsAnd the new failures with this PR as committed are now: Details |
The wrapper is supposed to catch connection errors and turn them into
xfail, buttest_rmdir_not_emptyalso catches connection errors because they areOSError(which is what it expect, thus the test still fails everything.)Instead, change the
xfail_on_github_rate_limitfunction to check for networking first, and mark everythingxfaildirectly.Also, remove the
_xfail_on_rate_limit_errorsfixture, which is redundant to the above and doesn't work fully because it only applies to the class's direct test methods and not its inherited test methods.