You have more chances to fix your code with AI

It used to be that writing code and tests was where you spent most of your development time budget. Depending on your role, PR reviews of other people's code might eat up a decent-sized chunk of time as well.

But however that initial writing turned out, that was usually the last time any code that got written received a detailed once-over. There might be an occasional refactor if things got ugly and you needed to extend that piece of code, but that was always a tough decision.

LLMs change the calculus on this idea, however. Suddenly, it’s not only perfectly reasonable but almost an obligation to give in-depth reviews of code using different lenses—a security lens, a performance lens, a duplication lens—on a regular basis to keep cruft and tech debt from accumulating.

Done right, these kinds of scans can be run as part of the process (see The 3x3 Rule of AI-Native Software Engineering), on recent commits overnight or on a semi-regular basis, and can keep the codebase headed in the right direction.

To make these useful, this idea needs to be combined with excellent code coverage in the test and eval suite so that changes can be made safely to the code without impacting behavior. E2E tests are particularly valuable for verifying that nothing went wrong at the user level.

These skills should generate an output artifact that can be reviewed and tracked over time—both in prose and via metric scripts.

Example: a weekly review

Given that most people are doing code reviews of PRs, let’s ignore the per-PR review skills and instead focus on something done on a weekly cadence.

Here are some examples of skills you can build to run, say, every Sunday evening and make part of your Monday morning manual review. These shouldn't make any code changes, just generate an output doc for manual review. Finding the right instructions to keep the review conclusions succinct and front-loaded will mean you actually read the output.

• api-surface-audit - a review of public API surfaces touched in the past week from the git log, a check of the exhaustiveness of their test coverage for all potential auth variations.
• code-churn - review files and areas of the application with the highest churn (use a script to generate candidates) - and then evaluate that churn: is it justified or is there a missing refactoring to keep changes more isolated.
• unit-test-coverage-review - a review of the current state of test coverage and what changed or dropped in the past week and why (script for coverage, prose analysis of the reasons)
• e2e-test-coverage-review - a review of the current state of end to end test coverage and whether all new added screens and interfaces are covered.
• test-suite-performance-review - a review of the performance of the test suite compared to the week before, a review of slow tests, and whether the slow tests are worth their weight or should be refactored (are the "load bearing" as my good friend Claude would say)
• eval-review - run the full eval suite and give a prose analysis of the results. Review code changes from the previous week and make sure the eval suite isn't being left behind. Depending on the level of LLM involvement in your application, your eval suite may be extremely important to keep on top of, or may be just a small piece of the puzzle.
• steering-doc-and-session-analysis - a review of the size of all the steering documents, their growth and a prose analysis of what's changed. This could be paired with an analysis of session breakdown to see if anything has shifted in terms of % of time spent in various subagents (implementation/review/fix/etc).

These reviews lean heavily on analysis of the test suite - as in many ways the test suite is your watch dog, keeping an eye on everything that's going on while the agents crank away.

I strongly recommend adding in a suite of review skills, both for individual commits and for the codebase at large that you run at a regular cadence to alert you to security, performance, code quality issues before they rear their ugly heads in production.