GitHub PR Merge Predictor
Betathink-reason-learn/github-pr-merge-predictorPredicts whether a pull request from an outside contributor will be merged, and explains what pulled the score up or down.
Reads a pull request the way a maintainer first sees it (title, description, linked issues, size, changed paths, commit messages and the diff) and predicts whether it will be merged.
Returns a verdict (likely rejected / needs work / uncertain / promising / likely merged), the merge probability next to the base rate, and the checks that pulled the score up or down. Use it to triage incoming PRs, to flag low-effort or AI-generated contributions, or to check your own PR before you open it.
Give a GitHub PR link and it is rebuilt as it was opened: the first title and description, and the diff at opening. Reviews, labels and later pushes are never read. Or send the title, description and diff inline to score a PR that doesn't exist yet.
Built with ThinkReasonLearn: plain-language checks answered by a language model and combined by a trained, auditable scorer. Trained on 2,842 PRs by outside contributors to 143 popular open-source repos (May 2025 to August 2026). On 36 repos it never saw it ranks merged PRs at 0.744 AUC; its top 10% of scores were merged 83% of the time and its bottom 10% 13%, against 46% overall.
Send the PR link as the user message, or a JSON object with title, body, diff. The reply is a JSON object.
Quality
- Merge AUC, unseen repos
- 0.744
- 90% CI 0.708-0.776
- Merge rate in top 10%
- 83%
- vs 46% overall
- Merge rate in bottom 10%
- 13%
Limits
Where the model was trained, and where to trust it less.
- Trained on PRs by outside contributors (not maintainers or bots) to popular open-source repos, May 2025 to August 2026. Maintainer and bot PRs carry a warning.
- It judges the PR as opened (title, description, linked issues, size, paths, commits, about 3.5k characters of diff). It does not see CI, reviews or later pushes, and does not run the code.
- It doesn't know a repo's own merge rate: a strong PR to a repo that rarely merges outside PRs can still be closed. Closed-without-merge in training includes superseded, duplicate and stale PRs.
- For a link, author standing means the author had a PR merged in that repo before. Inline input without author_association is scored with standing unknown.
- Long, fully structured descriptions (every change itemized, test commands listed, edge-case sections, the template completely filled) were merged less often in training. It is a common shape of AI-generated PRs, and the model counts it against a PR.
- Pulled up / pulled down are what the model noticed, not advice.
Input
| Field | Type | Description |
|---|---|---|
| pr_url | url | Pull request. Scored as it was opened. Leave empty to paste the PR below instead. |
| title | text | Title. Up to 1,000 characters. |
| body | text | Description. Up to 100,000 characters. |
| diff | text | Diff. Only the most informative ~3.5k characters are read; lockfiles and generated files are skipped. |
| repo | text | Repository. |
Output
The output object of POST /v1/predict. Over chat completions, the same JSON is the reply content.
| Field | Type | Description |
|---|---|---|
| verdict | string | likely rejected | needs work | uncertain | promising | likely merged |
| merge_probability | number | Probability the PR is merged. |
| level_merge_rate | number | How often held-out PRs with this verdict were merged. |
| base_rate | number | Share of all training PRs that were merged (0.46). |
| pulled_up, pulled_down | array | Checks {check, answer, effect}. What the model noticed, not advice. |
| pr | object | What was read: title, repo, url, files_changed, additions, deletions, author_had_merged_pr_before, diff_source, state. |
| warnings | string[] | E.g. the author is a maintainer, or the diff was excerpted. |
Call it
POST /v1/predict with your API key, or any OpenAI SDK pointed at our base URL.
curl https://api.thinkreasonlearn.com/v1/predict \
-H "Authorization: Bearer $TRL_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "think-reason-learn/github-pr-merge-predictor",
"input": {
"pr_url": "https://github.com/fastapi/fastapi/pull/16424"
}
}'