Greptile: Put Limits on Your AI Code Review Loop

Why it matters

An agent can produce a patch faster than a developer can establish whether that patch belongs in production. Connecting a coding agent to an automated reviewer addresses part of that gap: the reviewer raises an issue, the coding agent attempts a fix, and another review checks the result. The developer spends less time relaying comments between tools.

Micky describes using Greptile and a skill called greploop for that feedback cycle. He also reports that AI generated roughly 95% of his recent code. Those are useful details about his workflow, but the interview does not provide a controlled productivity comparison supporting the title’s “100x” claim.

I would evaluate this setup by the effort required to accept and maintain a change. A high review score is one input to that decision. This post adds an evaluation protocol to the discussion: define acceptance criteria first, preserve tests that can challenge the implementation, cap the loop, and measure the full cost of an accepted pull request.

Key points

• 01:05 The environment around the model affects its work. Micky presents Cursor as his main harness. File access, search, and command execution shape what an agent can investigate, so a model comparison should account for those capabilities.
• 04:21 Dependency source can resolve integration questions. The tool introduced here is Vercel Labs’ opensrc. It makes library implementations available for inspection when understanding the public interface alone is insufficient.
• 08:46 Context needs deliberate selection. The interview favors focused sessions. A practical interpretation is to provide relevant locations and questions, helping the agent investigate the task without carrying unrelated history.
• 10:26 A plan makes the implementation accountable. Smaller pull requests make it easier to spot scope expansion and check whether the resulting behavior matches the original request.
• 13:13 Fast implementation can accumulate duplicate behavior. Adding a Telegram integration illustrates how an agent may recreate existing functionality. A service layer can consolidate shared rules while leaving channel-specific responsibilities explicit.
• 16:59 Review becomes a separate stage. Micky introduces Greptile and its confidence scores. The findings behind a score provide actionable questions; the score alone offers limited information about correctness.
• 18:07 Greploop automates the return trip from feedback to patch. The coding agent reads comments, makes changes, and waits for another review. That removes repetitive coordination while creating a need for explicit stopping conditions.
• 21:25 A feedback cycle needs a verifiable destination. Tests and completion criteria guide successive attempts. Repeated execution cannot resolve an ambiguous requirement by itself.
• 23:35 Svelte reflects the guest’s own stack preferences. His explanation emphasizes familiar syntax and accessible source. That suggests evaluating how easily an agent can inspect a stack without assuming one framework wins everywhere.
• 25:45 Configuration expressed as code is easier to inspect alongside application logic. Convex is his backend example. He discloses that he works there, which is relevant context for the recommendation.
• 34:28 Releasing creates contact with real users. The discussion challenges endless launch delays. A bounded pilot translates that advice into a manageable way to collect feedback.
• 41:36 Dependencies are part of the security discussion. Micky suggests a fourteen-day waiting period for new packages. I would consider that a possible exposure-reduction policy, while still requiring other checks and a path for urgent security updates.
• 48:12 Experimentation supports learning. The closing discussion encourages curiosity about changing tools. A small experiment with recorded outcomes gives developers something concrete to use when choosing their next investment.

How to apply

Evaluate Greptile against behavior you defined first

Start with a modest change in an application you understand. For example, add an order webhook that tolerates repeated delivery without creating duplicate orders. A broad backend redesign introduces too many variables for an initial evaluation.

Write acceptance criteria before the agent implements anything:

• A valid event creates exactly one order.
• Replaying an event identifier preserves the existing order.
• Credentials for account A cannot create orders in account B.
• Invalid input leaves persistent state unchanged.
• A temporary failure allows a retry without duplication.

These requirements give you an evaluation surface beyond the reviewer’s own comments. The loop can address every finding and still miss a behavior that nobody asked it to check.

Supply references that match the installed version

Point the agent to the current handler, persistence function, nearby tests, and relevant dependency version. When an integration depends on implementation details, inspect the library source together with its versioned documentation.

The official opensrc project provides access to package and repository sources. Follow its current instructions rather than assuming the video’s commands and directory layout still apply.

Source shows implementation behavior. Documentation helps establish public contracts and supported usage. If they appear inconsistent, ask for the exact versions and files involved before accepting a workaround. Reading an internal function does not establish that it is a stable API your application should call.

Give the loop a budget and an exit report

The official greploop targets a 5/5 confidence score with no unresolved comments. The version checked for this draft also caps the cycle at five iterations. Read the skill definition.

For an initial pilot, I would add a time limit and a spending ceiling. Three rounds or twenty minutes could be starting experiment settings, adjusted to the task. These are proposed limits, not measured optimal values.

When any limit is reached, require a report showing the changes made, checks executed, and remaining findings. Stop earlier if successive rounds repeat the same issue, a fix requires changing the acceptance criteria, or the patch expands into unrelated functionality. Those conditions call for investigation before another attempt.

Keep tests capable of disagreeing with the patch

For a meaningful defect, request a reproducible input, the observed result, the expected result, and an associated test. Where practical, confirm that the test fails before the fix and passes afterward.

Review changes to assertions, integration fixtures, and mocks. A passing suite is less persuasive if the agent removed the scenario that exposed the problem. Tests generated from an implementation can also preserve its mistaken assumptions.

For the webhook example, cover both sequential replay and simultaneous delivery. Then inspect the stored result manually using synthetic data. That gives you an additional check outside the agent’s description of its own work.

Consolidate duplication when you can show it

If a webhook and another entry point implement the same domain rule, have the agent identify both locations and propose a shared function. Keep channel authentication and input translation explicit.

Move a substantial refactor into a separate pull request when necessary. The purpose of a small review is to let someone understand the behavioral change and its consequences. A cleaner directory structure does not compensate for an unreviewable patch.

Track the effort through acceptance

Compare a small sample of similar PRs with and without the automated cycle. Record elapsed time to acceptance, active human review minutes, iteration count, additional charges, findings confirmed as defects, and regressions discovered afterward. Treat the comparison as directional evidence: different tasks do not create a controlled benchmark.

Include subscriptions allocated to the pilot period, separately billed usage, and human time. Avoid counting usage twice when it is already covered by a subscription. A useful calculation is:

cost per accepted PR = (allocated subscriptions + additional charges + human time cost) ÷ accepted PRs

I would expand the workflow if it reduced human review effort without increasing regressions or making changes harder to understand. Final acceptance should account for requirements, test evidence, and observed behavior. Greptile’s score contributes to that decision alongside those checks.