fix: use REST API for user search to fix incorrect sort order#113
Open
unhappychoice wants to merge 1 commit intoashkulz:masterfrom
Open
fix: use REST API for user search to fix incorrect sort order#113unhappychoice wants to merge 1 commit intoashkulz:masterfrom
unhappychoice wants to merge 1 commit intoashkulz:masterfrom
Conversation
The original implementation uses GraphQL search with sort:followers-desc in the query string, but GitHub does not support the sort qualifier for user search. This causes non-deterministic results and missing users. This commit replaces the user search with a two-step approach: 1. REST API GET /search/users?sort=followers (officially supported) 2. GraphQL user(login:) batch queries for detailed data Other changes: - Add MinFollowers field to UserSearchQuery and QueryPreset - Set minFollowers=1000 for the worldwide preset to avoid incomplete results - Exclude the "claude" account (Anthropic co-author bot) which causes GitHub GraphQL to return 504 on contributionsCollection - Add exponential backoff retry for GraphQL batch errors - Add rate limit delays between API requests
Author
|
BTW, thank you for maintaining this project! I really appreciate the work you've put into it. |
Owner
|
@unhappychoice thank you for the fanastic analysis! I'm a bit tied up right now, I'll review this on Wednesday. I'm standing on the shouders of giants -- @lauripiispanen did most of the work, and I'm just keeping the lights running 🙂 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #112
Summary
The GraphQL user search uses
sort:followers-descin the query string, but GitHub does not support thesort:followersqualifier for user search. This causes non-deterministic results and missing users in the rankings.Changes
Replaces the single GraphQL search query with a two-step approach:
GET /search/users?sort=followers&order=descfor collecting user logins — the REST API officially supportssort=followersand returns correctly sorted results with no duplicatesuser(login:)batch queries (10 users/batch) for fetching detailed data (contributions, organizations, follower counts)Other changes
MinFollowersfield added toUserSearchQueryandQueryPreset— the worldwide preset usesminFollowers: 1000to narrow the search scope. Without this filter, the REST API search may hit GitHub's time limit and return partial results (the response'sincomplete_resultsfield would betrue), since worldwide has 160M+ users.minFollowers()fix — computes the actual minimum across all users instead of using the last user's follower countclaudeaccount — the Anthropic Claude Code co-author account causes GitHub GraphQL to return 504 oncontributionsCollectiondue to its massive co-authored commit volumeTesting
Verified with production settings (
consider=1000,amount=256):Files changed
github/github.goSearchUsers()to use REST + GraphQL two-step approachpresets.gominFollowersfield toQueryPreset; setminFollowers: 1000for worldwidetop/top.goMinFollowerstoOptionsstruct; pass through toUserSearchQuerymain.gopreset.minFollowersand pass totop.Options