Should the AI reviewer block merges?

Start with non-blocking comments so the team can judge quality before trusting it. Once you have tuned the prompt and the false-positive rate is low, you can make specific categories (such as detected secrets) a required check. Never let it be the only reviewer.

How do I keep the diff under the context limit?

Send only the changed hunks, not whole files, and skip lockfiles, generated code, and binary blobs. For very large PRs, review file by file in parallel and merge the comments, or summarise per file before a final pass.

How do I stop it nitpicking style?

Tell it explicitly in the prompt to ignore formatting that a linter or formatter already handles and to focus on correctness, security, and clarity. Pair the reviewer with your existing linters rather than duplicating them.

Where do I store the API key?

As an encrypted GitHub Actions secret (such as OPENAI_API_KEY), referenced via the secrets context in the workflow. Never commit a key to the repository — Actions injects secrets into the runner environment at job time.

How much does an automated reviewer cost?

Each PR review is one or a few model calls over the diff, typically a fraction of a cent to a few cents depending on diff size and model. Skipping generated files and reviewing only changed hunks keeps both cost and noise down.

How to Build an AI Code Review Tool

What you are building

An AI code review tool is a GitHub Action that fires whenever a pull request is opened or updated, reads the diff, asks a language model to review it against your team’s standards, and posts the feedback back onto the PR as comments. It does not replace human review — it catches the obvious issues (missing error handling, a leaked secret, an off-by-one, an unclear name) before a human looks, so reviewers spend their attention on design and intent.

How it works

The workflow runs on the pull_request event. A step checks out the code and extracts the diff of changed files against the base branch — typically with git diff and a filter that drops lockfiles, generated files, and binaries to save tokens and noise. That diff is dropped into a review prompt alongside your rules (“focus on correctness and security, ignore formatting the linter handles, output JSON”). The model is called with your API key, supplied as an encrypted Actions secret. Its structured response is then posted back using the GitHub REST API — either as inline comments anchored to specific lines, or as a single review summary.

The quality of the reviewer lives almost entirely in the prompt. Asking for structured output (a JSON array of {file, line, severity, message}) makes the response easy to turn into inline comments and easy to filter by severity. The builder below assembles a complete review prompt from your rules and a pasted diff so you can see exactly what your Action would send.

Tips for a reviewer people actually keep

Start non-blocking — let it comment for a week and read the noise before you make any check required. Tell it to ignore anything a formatter or linter already enforces, or it will drown real findings in style nags. Cap the diff size and skip generated files so you stay under the context limit and under budget. Ask it to rate severity so you can surface only the important comments. And always keep a human in the loop: the AI is a fast first pass, not the approver.