GitHub Contributor Rating
Betathink-reason-learn/github-contributor-ratingRates how likely a GitHub user's pull requests to other people's repos are to be merged, from their public track record.
Rates a GitHub contributor from their public history: how their pull requests to repositories they don't own have gone, how they work across projects, and their standing in the repo they're contributing to.
Returns a rating (likely low-effort / unproven / mixed / solid / established), the probability that a PR of theirs to an outside repo is merged next to the base rate, the reasons that pulled the rating up or down, and the profile it read. Use it to triage a queue of outside PRs, or alongside the GitHub PR Merge Predictor, which judges the PR itself.
Give a username to rate them as of now, or a PR link to rate its author as of when that PR was opened (the PR itself is left out).
Built with ThinkReasonLearn: plain-language checks answered by a language model plus a few track-record numbers, combined by a trained, auditable scorer. Trained on 3,070 closed PRs by outside authors to 86 popular repos (May 2025 to September 2026). On 903 authors it never saw it ranks merged PRs at 0.78 AUC: 95% of its top tenth were merged against 17% of its bottom tenth.
Send a username or PR link as the user message. The reply is a JSON object.
Quality
- Merge AUC, unseen authors
- 0.78
- Merge rate in top 10%
- 95%
- vs 58% overall
- Merge rate in bottom 10%
- 17%
Limits
Where the model was trained, and where to trust it less.
- Trained on closed PRs by outside contributors to popular open-source repos, May 2025 to September 2026. Smaller or private projects may behave differently.
- It rates the person, not the PR: a strong contributor can still send a PR that should be closed, and a newcomer's first PR can be excellent. For people with fewer than 5 earlier PRs it is much less sure.
- Whether a PR is merged also depends on the repo; some popular repos close most outside PRs.
- It reads only public GitHub activity, so people who work mostly in private look newer than they are. Followers and profile fields are read as they are today.
- Accounts GitHub hides from search can't be read; their rating rests on an empty history.
- Without a target repo, it assumes they have no merged PRs in the repo in question.
Input
| Field | Type | Description |
|---|---|---|
| github_username | text | GitHub username. |
| pr_url | url | Or a pull request. Rates its author as of when the PR was opened. |
| target_repo | text | Repository. The repo they're contributing to. Defaults to the PR's repo. |
Output
The output object of POST /v1/predict. Over chat completions, the same JSON is the reply content.
| Field | Type | Description |
|---|---|---|
| rating | string | likely low-effort | unproven | mixed | solid | established |
| merge_probability | number | Probability a PR of theirs to a repo they don't own is merged. |
| level_merge_rate | number | How often held-out PRs by people at this rating were merged. |
| base_rate | number | Share of all training PRs that were merged (0.59). |
| pulled_up, pulled_down | array | Up to 5 reasons each: {check, answer, effect}. |
| profile | object | What was read: login, as-of date, account age, followers, earlier outside PRs (merged / closed unmerged), merge rate, recent PRs, PRs and merges in the target repo. |
| warnings | string[] | E.g. no target repo, little history. |
Call it
POST /v1/predict with your API key, or any OpenAI SDK pointed at our base URL.
curl https://api.thinkreasonlearn.com/v1/predict \
-H "Authorization: Bearer $TRL_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "think-reason-learn/github-contributor-rating",
"input": {
"github_username": "sindresorhus"
}
}'