You're Already Using an AI Coding Agent But Still Reviewing PRs Manually

Here's the situation: Your team is using AI coding agents. They're writing features, fixing bugs, and opening pull requests. But somehow you're still stuck reading through every diff, manually deciding what's safe to merge. The irony is real — you've automated the writing but kept the gatekeeping manual.

This article shows how to close that loop. Not with more tools, but with a system that gates merges based on verifiable evidence — and gets smarter every time something goes wrong.

The Basic Flow

Here's how automated PR review should work when you're using AI coding agents:

This is a loop, not a pipeline. The remediation agent can trigger new commits, which trigger new preflight checks, which trigger new reviews. The cycle continues until the PR meets all requirements — or a human intervenes.

The Four Things That Actually Make It Work

This sounds straightforward, but most teams fail at implementation. Here are the four things that actually make it work in practice:

1. One Contract File Owns the Rules

Risk tiers, required checks, and documentation rules all live in a single JSON file. No drift between scripts, workflow files, and policy documents. When you change a rule in one place, it updates everywhere.

Without this single source of truth, you end up with scattered configuration that slowly diverges. One workflow says tests are required for backend files. Another says only security scans matter. The preflight check uses a third definition of "high-risk." The result is confusion and gaps that bugs slip through.

2. Preflight Runs Before CI

You don't waste compute on a PR that's already blocked. The preflight gate is fast and cheap — it's just analyzing file paths and checking against your contract file. The expensive stuff (builds, test suites, security scanners) only spins up if the gate passes.

This is especially important when AI agents are opening multiple PRs per day. Running full CI on every automated PR that gets blocked by a simple policy violation is a waste of time and money. Preflight catches the obvious issues before they trigger the heavy machinery.

3. SHA Discipline Is Non-Negotiable

Every check must be tied to the current PR head commit. If the branch gets a new push, everything reruns. Stale "clean" evidence is treated as no evidence.

This seems obvious, but many teams get it wrong. They allow checks to pass if they succeeded on any commit in the PR's history. That's a security hole — an attacker could push a malicious commit, get it reviewed and approved, then amend it with a payload that bypasses the original review.

SHA discipline means commit-a1b2c3 passing tests doesn't help commit-d4e5f6. Every new push resets the board.

4. One Bot Writes Rerun Comments

Multiple workflows trying to trigger reruns causes race conditions and duplicate comments. Pick one workflow as the canonical requester and deduplicate by commit SHA.

When you have several GitHub Actions workflows that can all comment "/rerun" on a PR, they step on each other. Two workflows notice a new commit, both post the comment, and now you're running checks twice. Or worse, the deduplication logic fails and you end up in an infinite rerun loop.

Designate one workflow as the rerun authority. Others signal it to trigger, but only one bot actually posts the comment.

What You Get Out of It

When this system is working, you get something most teams never achieve: fully machine-verifiable merges. Here's what that means in practice:

Merges Are Fully Machine-Verifiable

Every merge decision has an audit trail. You can trace exactly which checks passed, on which commit, and see the evidence. There's no "looks good to me" rubber-stamping — there's either proof that the required checks passed, or the PR doesn't merge.

Production Bugs Become Test Cases

When a bug slips through, you don't just fix it and move on. You convert it into a permanent harness test case so the same gap can't happen twice. The system gets more reliable over time because failures compound into coverage rather than being forgotten.

This is the self-improving aspect. Each production incident becomes a new rule in your contract file, a new check in your preflight gate, or a new pattern your review agent looks for. The system learns from its mistakes.

Humans Focus on the Right Problems

You're no longer reviewing every diff just in case. The automated flow catches the routine issues — policy violations, obvious bugs, security red flags — and only escalates to humans when it encounters something it can't handle.

Your attention goes to the edge cases, the ambiguous requirements, the decisions that actually require judgment. Not the thousandth implementation of the same validation logic that your remediation agent could fix in 30 seconds.

Getting Started

You don't need to rebuild your entire CI/CD pipeline tomorrow. Start with these three steps:

1. Create your contract file — Define what files are high-risk, what checks are required for each tier, and document the rules. This is the foundation.
2. Build a preflight gate — Add a lightweight workflow that runs before your main CI and checks against your contract file. Block PRs that don't meet requirements.
3. Add SHA discipline — Make sure every check records the commit SHA it validated, and reruns automatically when the PR head changes.

Once those are in place, you can layer on the code review agent and remediation automation. But even with just the preflight gate and SHA discipline, you've already eliminated the most common failure mode: merging code that was never actually verified against your current rules.

FAQ

Why do I still need to review PRs manually if I'm using AI coding agents?

Most AI coding agents generate code but don't have the authority to merge it. You're still the gatekeeper, reading through diffs and making decisions. This article shows how to close that loop with automated PR review workflows that gate merges based on verifiable evidence tied to each commit.

What is a preflight check in code review?

A preflight check is a lightweight analysis that runs before expensive CI operations like builds, tests, and security scans. It evaluates which files changed, assigns them a risk tier (high or low), and determines what evidence is required before the PR can merge. This prevents wasting compute on changes that are already blocked.

How does SHA discipline work in automated PR reviews?

SHA discipline means every check must be tied to the current PR head commit. If the branch gets a new push, everything reruns. Stale evidence from an older commit is treated as no evidence. This ensures that what passed for commit-a1b2c3 doesn't count for commit-d4e5f6.

What happens if the automated review finds an issue?

When the code review agent finds something actionable, a remediation agent can automatically patch the code, push a fix commit, and trigger the review loop again. This continues until all issues are resolved or a human intervenes.

How does the system learn from production bugs?

Every production incident gets converted into a permanent test case or rule. This could mean adding a new pattern to your contract file, creating a new automated test, or teaching your review agent to look for a specific issue. The system's coverage expands with each failure.