Write your own call down before you read the AI’s

I think the riskiest part of AI recommendations is the decision you’d swear was yours.

Anthropic’s alignment team just put numbers on it. In a study of how AI agents shape experts’ judgment calls, analysts used an agent to work through a business decision: whether a fictional farming co-op should expand one of its programs. The agent was set to lean one way on each of four analysis choices. 86% of the analysts’ choices matched that lean, where chance would sit near 50%. Most of them repeated the agent’s arguments. Only 15% gave it any credit. And more than half still called the final recommendation more theirs than the agent’s.

The researchers are careful about the limits. They made the agent’s leans stronger on purpose, it was one task with one model, and real effects may be smaller. The agent also helped. People using it made far fewer serious errors than people working alone, though the two groups weren’t evenly matched on experience. So the lesson is to use it deliberately.

Why does this matter for an owner?

Because the decisions that shape your business are the ones you’re most likely to run past an AI for a second opinion. Pricing. A big quote. A hire. Whether to keep a difficult client. If the tool leans, odds are you lean with it.

My line on AI hasn’t changed. AI does the work. People set the rules and decide where it stops. This study adds one thing to that. Deciding where it stops includes deciding where it doesn’t get a vote until you’ve voted first.

It’s a different problem from a person approving AI output on autopilot. There, nobody is really checking. Here, you’re checking carefully, and your judgment is still being steered. It’s also why I’d keep one rep a week for your own thinking, with no tool in the room.

How do you keep the big calls yours?

Name the decisions that stay with you, then write your own lean down before you open the AI’s answer. One line is enough. Afterward, compare. If you changed your mind, you’ll know exactly why.

• Name them. The handful of decisions where your judgment is the product. Most owners can list them in a minute.
• Write your lean first. One sentence, before you read a word of the recommendation.
• Ask for the other side. Have the AI make the strongest case against its own answer, so you see both before you choose.
• Note what moved you. If you switch, write the reason in a line. “The numbers showed a gap I missed” is a good reason. “It sounded right” isn’t one.

Take a Cochrane property manager deciding on rent increases across a building, as an example. Before she opens the AI’s analysis, she writes one line: hold rents this year, because renewals matter more than a few extra dollars a door. Then she reads the recommendation. If it changes her mind, it had to earn it.

This is the same idea as the stop rule in an AI agent job description, pointed at your own head instead of the agent’s.

My take

Keep using the agent. The study suggests it catches mistakes people miss. Just make your own call first on the decisions that carry your name. If your leadership team needs to agree on which calls stay human, that’s what my executive AI workshops are for, and if you want the bigger picture on where agents fit, start with AI agents for business, role by role.

AI recommendations, the questions owners ask

Write your lean down before the next big AI answer. I share what I’m testing every Friday in the AI Newsletter with ZAK.