Code Review Best Practices: A PR Checklist and a Mutation Testing Example

You've got a PR open and two questions: what should the reviewer look at, and can you trust a green test suite? Below: what Google's review guide, the Cisco study and Microsoft's research say, a code review checklist you can paste into a PR template, and a Stryker run where tests with 100% coverage let six bugs through.

Pull request size: what the sources say

• Google says about 100 lines is usually a reasonable change and 1,000 is usually too large, but defers to the reviewer, and adds that spread matters: 200 lines in one file may be fine, across 50 files it's usually too big. Their reasons are practical: "It's easier for a reviewer to find five minutes several times to review small CLs than to set aside a 30 minute block to review one large CL," and in big changes "important points get missed or dropped."
• Cisco's study, as summarised by SmartBear, recommends reviewing no more than 200 to 400 lines at a time, over 60 to 90 minutes, which "should yield 70-90% defect discovery." The same summary says lightweight review took under 20% of the time of formal inspection and found just as many bugs. That is a vendor's summary of the study; I did not locate the original Cisco/SmartBear data, so treat 70-90% as vendor-reported.
• Google's approval standard is whether the change definitely improves code health, even if it isn't perfect. That's an argument against blocking on taste.

A decision rule that doesn't depend on a line count: split the PR when it crosses two concerns (a refactor plus a behaviour change, a rename plus a bug fix), because then the reviewer can't hold one question in their head. A 600-line mechanical rename is one concern; a 120-line change that fixes a bug and reshapes an API is two.

What code reviews actually find

Microsoft researchers observed, interviewed and surveyed developers and managers, and classified hundreds of review comments. The abstract says finding defects remains the main motivation, but "reviews are less about defects than expected and instead provide additional benefits such as knowledge transfer, increased team awareness, and creation of alternative solutions to problems." It also says "code and change understanding is the key aspect of code reviewing."

A second study, Rigby and Bird's comparison of review at Google, Microsoft and AMD, found that peer review increased a developer's familiarity with distinct files by 66% to 150%, depending on the project.

The practical upshot: don't count on the reviewer to be your bug net. Their strength is design, naming, "why is this here" and missing cases, so write a clear description with one concern per diff, and leave mechanical bugs to tests and linters.

Code review checklist for pull requests

Adapted from the areas in [Google's "What to look for in a code review"](https://google.github.io/eng-practices/review/reviewer/looking-for.