LG Salles

AI, sales, marketing and development

Desenvolvimento

Claude agent harness: test before adding skills

November 19, 2026

Video by David Ondrej · Original title: “Matt Pocock’s Agentic Engineering Workflow (just copy him)”Subscribe to the channel

Why it matters

A coding agent can become more elaborate without making development easier. Add enough instructions and it may produce longer plans, more files, and increasingly polished explanations. The useful question is how much work remains before you can trust and ship the change.

Matt Pocock’s conversation with David Ondrej offers a practical starting point: keep engineering judgment with the developer, turn repeatable practices into small procedures, and inspect the environment surrounding the model. My approach would be to treat every added skill as a hypothesis. Run comparable tasks, examine the resulting changes, and keep the procedure when it improves the complete delivery process. The experiment below gives that idea a concrete shape without assuming the interview’s productivity claims apply to every team.

Key points

  • 00:32 Tactical and strategic programming provide a useful distinction between executing changes and deciding how the system should evolve. The speakers’ large productivity claims are personal assessments, rather than results from a controlled comparison.
  • 06:08 The teach skill connects learning to a specific mission. Its meaningful outcome is whether the learner can perform a relevant task afterward; a persuasive lesson alone cannot establish that.
  • 11:38 Learning progress is stored in local files. This makes the teaching process inspectable and creates an opportunity to correct an inaccurate record of what the learner understands.
  • 18:19 The grill-me procedure interviews the user about an idea. Its contribution should be judged by consequential decisions resolved, rather than the length or apparent sophistication of the conversation.
  • 25:06 Sandcastle supports running agents in isolated environments. That architecture helps bound delegated work, while the actual protection still depends on how access and credentials are configured.
  • 27:55 Pocock emphasizes the harness around the model. Tools, instructions, permissions, and feedback mechanisms are all variables a developer can adjust and evaluate.
  • 32:19 A codebase that is easier to change may require less exploration and fewer failed attempts. Whether this also makes a cheaper model viable needs testing against the repository’s real tasks.
  • 45:04 A task queue gives unattended work a clear unit of progress. Each item needs an observable outcome and an endpoint so completion can be assessed.
  • 51:07 Review produces information about the development process as well as the code. Repeated mistakes can expose missing checks or confusing instructions that deserve attention.
  • 1:00:33 The closing recommendation is to strip back configuration and restore useful additions selectively. I would run that experiment in a disposable environment with the original setup preserved.

How to apply

Establish a Claude agent harness baseline

The harness is the operating environment around the model: the runner, available tools, instructions, permissions, and feedback it receives. A skill changes part of that environment. To find out whether it helps, keep the other variables reasonably stable.

Start with a repository you understand and a small, observable change. An export bug is a useful example: a CSV file mishandles Brazilian currency values such as 1.234,56. Define expected behavior for that value, an empty field, and a negative amount before asking the agent to implement anything.

Create two disposable copies from the same commit. Give one the minimal configuration and add the procedure under evaluation to the other. Keep required access controls and team policies in both. Record which optional instructions differ, rather than relying on your memory of the setup.

Use fresh sessions and record the model, agent version, loaded instructions, and verification commands. Pin the skill revision when practical. Otherwise, a background update can quietly change what you are comparing.

Choose a procedure that addresses an actual uncertainty

For an idea that still needs clarification, the current grill-me documentation provides this installation command, including its grilling dependency:

npx skills@latest add mattpocock/skills --skill=grill-me --skill=grilling

Invoke /grill-me in a fresh conversation. The collection has changed since the June interview, so use the current official instructions when installing it.

Give the interview a concrete decision to resolve. Should an export include every customer or only the currently filtered results? Explain who uses it and what they do with the file. Write down the decision when the conversation ends, including any assumptions that still need checking.

Some uncertainty requires an artifact. If you cannot choose between two interface arrangements without seeing them, build a small disposable prototype and return with observations. Rephrasing the question repeatedly will not create that missing evidence.

For a knowledge gap, separate learning from feature work. The teach documentation describes a dedicated workspace containing the mission, lessons, and learning records. Give it a mission such as learning to diagnose authorization failures, then check progress with an exercise you can repeat independently. See the official teach documentation.

Measure the full delivery process

Try three representative tasks: a bug fix, a small feature, and an internal change. Run each with and without the relevant procedure, always restoring the starting state. Repeat surprising results and alternate which configuration goes first to reduce the advantage created by your own growing familiarity.

Record a few comparable observations:

| Measure | Evidence to collect | |---|---| | Acceptance | Previously defined examples that pass or fail | | Intervention | Times you had to correct the agent’s understanding | | Review effort | Minutes spent inspecting and repairing the change | | Scope discipline | Unnecessary changes outside the request | | Consumption | Reported cost, available token metrics, and elapsed time |

Include interview time in the total. An extra ten minutes of clarification can be worthwhile when it saves substantial rework. The same interview can become overhead when the task already has precise requirements.

A subscription also has limited capacity. Where task-level billing is unavailable, record the usage information you can actually observe. Avoid manufacturing an exact dollar cost from incomplete data.

This small trial supports a local decision about your workflow. It does not establish that a skill will help every repository, language, or developer.

Define what an unattended agent must return

Write the requested behavior, scope boundaries, verification method, and stopping conditions before delegating the implementation. For the CSV example, ask for the fix, automated examples covering the agreed formats, and a sample export using fictional data.

Set a time or spending limit using the controls your runner supports. If the agent cannot reproduce the problem, or discovers that the fix requires a product decision, have it return the investigation and the unresolved question. That is a useful endpoint for the task.

Give every retained skill a reason to stay

After the trial, connect each retained procedure to an observed improvement: clearer scope, stronger verification, or less repair during review. If the benefit is inconsistent, remove it from the experimental configuration and reconsider it when a different need appears.

I would also track recurring review comments. If reviewers repeatedly explain the same export rule, a concise project instruction may help. If they repeatedly struggle to understand the implementation, the code’s structure may need work. Choose the intervention that addresses the evidence, then repeat the comparison.

A useful configuration should make the final change easier to assess and maintain. That is the standard I would use before adding the next skill.

Resources mentioned

  • Matt Pocock Skills · Repositório oficial; inclui teach, grill-me e outros procedimentos. O catálogo mudou desde a gravação.
  • Sandcastle · Biblioteca para executar agentes em ambientes isolados; modelos e infraestrutura podem gerar custos.
  • Claude Code (paid) · Ferramenta utilizada no fluxo demonstrado; acesso e cobrança dependem da modalidade contratada.
  • AI Hero — Skills · Documentação e catálogo do autor; oferece também curso por e-mail mediante cadastro.
  • A Philosophy of Software Design (paid) · Livro de John Ousterhout citado na discussão sobre programação estratégica; página oficial do autor.
  • Pro Git · Livro indicado na demonstração do teach; leitura online gratuita, com tradução para pt-BR.
  • Wispr Flow (paid) · Ditado usado por Matt; oferece modalidade gratuita e planos pagos.
  • Superpowers · Coleção apresentada como contraste à preferência de Matt por procedimentos acionados pelo usuário.
  • Ralph — artigo original de Geoffrey Huntley · Referência discutida no trecho sobre loops e trabalho fora do teclado; a execução dos agentes pode gerar custos.