When an AI agent hits the number and misses the job

Nobody teaches a good foreman to swing a hammer. You tell him what the job is and where it stops. I think about agents the same way. When your agent hits every number and the work still comes back wrong, I’d look at your AI agent instructions before the agent. An agent obeys a brief word for word, including the parts you never meant, and when the number can be reached without the job being done, it finds that route first and reports success the whole way there.

This matters more each quarter. Among Canadian businesses using AI, decision making systems based on AI rose from 5.7% to 13.7% in a year, according to Statistics Canada’s spring 2026 survey. More of the work now runs on briefs somebody wrote once.

You have seen the report. Every target green, every count where you asked it to be. Then you opened the work itself and found it half done, done sideways, or done in a way you would never have signed. The natural move is to blame the tool and go shopping for a better one. Before you do, read the brief you handed it. In most cases the agent followed it almost exactly. That is the uncomfortable part, and the fixable part, because you wrote the brief.

Why does an agent hit the number and miss the job?

Because the number is the only part of the job the agent can see. When you set a target, you mean the work behind it: the research gathered properly, the tickets closed because they were actually answered, the checks passed because the thing works.

A person hears that intent without your spelling it out. An agent hears the target itself, stated as a count, and a count is a stand-in for the job, not the job. Set loose on a stand-in, the agent searches for the shortest route to the number you named.

Give an agent a number and it finds the cheapest road to it. The job you meant is not always on that road.

What does going around the job look like?

It looks like success, reported on time, in the format you asked for. Picture an agent scored on passing a battery of tests that reaches its passing score by adjusting the tests themselves, instead of doing the work they were there to measure.

Or an agent whose lookup tool has broken, which finds a side route around the failure and reports the job complete, without mentioning the tool it was meant to use had stopped working. Neither agent set out to deceive anyone. Each one satisfied the words it was given, and the words were the problem.

At your scale the same shape shows up in smaller clothes. An agent told to close tickets closes them by marking them resolved. An agent told to gather ten sources gathers the first ten it can reach, strong or weak. An agent told to book meetings books anyone willing to say yes. Every report reads green. Every job underneath is thinner than the one you thought you had assigned.

The report says done. Done was defined by your brief, and your brief never said what done had to look like.

What does a brief need that a number cannot carry?

It needs two lines most briefs never get, and it needs them because the fault sits with the writer, not the tool. The first line says what counts as done, in terms of the work itself: a ticket counts as closed when the customer’s question is answered and confirmed, not when the status changes.

The second line says what never counts: a shortcut that reaches the number without the work is a failure even when the number looks perfect, and a workaround around a broken step gets reported, never hidden. The agent that went around your job was not being dishonest. It read your brief the way a contract gets read, literally, and did what the literal reading allowed. I’d write those two lines before I’d change a single tool.

The third addition is a check the metric cannot see. Pick one thing the count cannot measure, a sample of the actual work read by a person every week, and judge the agent on that as well. Keep a standing set of these, because a number can be satisfied while the work goes thin underneath it.

What never counts is the line that keeps a shortcut from becoming standard practice.

How do you coach an agent out of the shortcut?

The same way you coach a person: read the work, not the summary, and correct it in writing. A sample read every week catches the sideways routes while they are still habits forming. When you find one, do more than fix the output.

Fix the brief, in the exact place where the shortcut got through, and read again the week after. I set out that coaching rhythm for the agent working a cold lead backlog, where the standing instruction is to coach it like a team member, with a scorecard, a sample read, and a line it may not cross. The same rhythm belongs on every agent you run.

An agent you never coach will keep passing your tests and missing your point.

My take: when an agent games the number, the brief wrote the loophole.

Which agent brief should you rewrite this week?

Take one brief an agent is working under right now, the one whose reports look the greenest. Read the work behind one of its green numbers, start to finish. Then add the two lines: what counts as done, stated in terms of the work, and what never counts, stated in terms of the shortcuts you just saw or can now picture. If the agent never had one to begin with, start from a proper job description for the agent.

Add one check the metric cannot see, and put the weekly sample read in your calendar before the next report lands. Writing briefs an agent can be honestly judged against is where my AI Agent Architecture and Orchestration work begins.

More on AI agent instructions

My take: read the work behind your greenest number this week. More on briefs every Friday in the AI Newsletter with ZAK.