The Cost Sheet

_{WHY THIS EXISTS}

The wins are real. The costs are also real.

Every AI creator on the internet shows you the wins. The 90-minute build. The agent that closes deals. The stack that runs itself. The cost almost never makes it into the post.

I think that’s a problem. Not because the wins are fake — most are real. Because the wins without the costs are misleading. They make AI work look like a one-way escalator. It isn’t. This guide is the more honest version — five mistakes from a year of building, what they cost, what I’d have done differently, and the questions I now ask before any build starts.

“The mistakes weren’t technical. Not one. They were all upstream of the code.”

What you’ll have when you’re done

• Five mistakes broken down with cost, root cause, and the rule that came out.
• The pre-build checklist I now run before any internal AI work begins.
• The pattern that connects all five — and what it means for your work.

_{MISTAKE 01}

Building before deciding.

COST: About 3 weeks of dev time, mid-2025.

What happened

I asked a developer to start building an internal tool before I’d written down what success looked like. We had a vibe , “something that helps with onboarding” and we had a developer with capacity. We started building. Three weeks in, the tool worked perfectly. For a job nobody actually had. We shipped it. Nobody used it. We deprecated it within a month.

Root cause

I confused having a developer available with having a problem to solve. Capacity is not a strategy. The team had energy and tools, so we built. But the underlying job wasn’t real. Nobody on the team would have personally chased that workflow if I hadn’t directed it. The tool was fine. The job wasn’t.

THE RULE THAT CAME OUT
A vibe is not a spec. The first hour of writing the brief saves a month of building the wrong thing.
Now: no build starts until the brief is signed off and the user is named.

_{MISTAKE 02}

Optimising the model before the brief.

COST: Two weeks I’ll never get back.

What happened

A customer-facing agent kept giving inconsistent replies. I assumed it was a model problem. I switched models. Tweaked parameters. Added more instructions on top of vague instructions. Performance got worse, not better. After two weeks of fiddling, I rewrote the original brief from scratch. Took 90 minutes. The cheapest fast model worked perfectly with the new brief.

Root cause

When something is unclear, the temptation is to add more, like more rules, smarter models, more guardrails. But complexity stacked on top of vagueness produces only more vagueness. The original brief was unclear about what ‘good’ looked like, and no model can compensate for that. The harder I tried to fix it at the model layer, the further from the actual problem I got.

THE RULE THAT CAME OUT
When an agent is misbehaving, the model is almost never the problem. The brief is. Always rewrite the brief before touching anything else. Now: I refuse to debug an agent without first re-reading the brief in full and asking ‘would a smart human do the right thing if given exactly this?’

_{MISTAKE 03}

No agent owner.

COST: An ops workflow that quietly stopped working for ~6 weeks before anyone noticed.

What happened

We deployed an agent into a recurring ops task. It worked great. Then it broke silently. The underlying input format had changed, and the agent started producing outputs that were technically valid but operationally wrong. Nobody had been formally assigned to monitor its outputs, so we found out only when a downstream process surfaced the issue six weeks later.

Root cause

We treated the agent like a tool. Tools don’t need owners, they sit there until used. But agents produce work autonomously. Work without an owner has no quality bar, no review cadence, and no accountability. We built the agent and walked away. The agent kept going, but the standard quietly drifted because nobody was holding the line.

THE RULE THAT CAME OUT
Every agent needs a human on the hook. Not a champion. Not a fan. An owner who reviews outputs on a schedule and is accountable when they degrade. If you can’t name the owner, don’t deploy the agent. Now: no agent goes live without a named owner and a review cadence on their calendar.

_{MISTAKE 04}

Tool sprawl.

COST: Real money on five separate AI tools doing roughly the same job for different team members.

What happened

We didn’t sit down and decide our content stack. Each person picked their own tool to help with writing, drafting, or editing. By the time we audited, we had five active subscriptions, three different brand voices in our published content, and zero consistency. The tools were each individually fine. The stack was a mess.

Root cause

We optimised locally, not globally. Each person picked the best tool for their workflow, but no one was responsible for the company’s overall stack. The result was duplication, drift, and a spend rate that didn’t match any clear value. Local choice without central discipline produced a graveyard.

THE RULE THAT CAME OUT
Tools without a stack are just spending. One job, one tool. If a tool tries to enter the stack, an existing tool has to leave. Stack discipline is more valuable than tool quality. Now: we run a quarterly Tool Graveyard. Anything that fails the audit gets killed publicly, in the team channel.

_{MISTAKE 05}

Building agents for the wrong problem.

COST: An entire agent built to “automate” something that was happening 4 times a month.

What happened

We built an agent to handle a workflow we believed was high-volume. We never measured. Three weeks into the build, I checked the logs. The workflow ran 4 times last month. Building the agent took longer than the next five years of doing it manually. We shipped it anyway out of pride. We deprecated it three months later when nobody could remember why we’d bothered.

Root cause

We assumed frequency. We never verified it. The instinct to automate is strong, especially when automation is your business. But instinct without measurement leads to building expensive answers to small problems. The right question wasn’t ‘can we automate this?’ It was ‘is this worth automating?’

THE RULE THAT CAME OUT
Before automating anything, count. How often does this actually happen? Calculate the payback period. If the answer is ‘more than 18 months from now,’ do it manually and revisit later. Now: every build proposal includes the actual frequency of the work and the projected payback period. Builds with payback longer than a year need explicit founder approval.

_{THE PRE-BUILD CHECKLIST}

Five questions before any internal AI build.

Direct consequence of the five mistakes above. We now run this checklist before any internal AI build starts. It takes 30 minutes. It saves weeks.

01. Is the brief written and signed off?

Not a vibe. Not a Slack thread. A document with role, inputs, standards, edge cases, and escalation. Signed off by the owner.

02. Is the user named?

A specific person who will use the output of this work, on a stated cadence. If “the team” is the user, no one is.

03. Is the owner named?

The human accountable for the agent’s outputs. With a review cadence on their calendar before deploy.

04. Is the frequency measured, not assumed?

Actual numbers from logs or counts. If the work happens fewer than 10 times a month, the build needs explicit founder approval.

05. Is the kill criterion defined?

What condition retires this build? Without one, builds stay in the stack forever, becoming part of the graveyard.

THE HONEST READ
Every mistake on this list passed an emotional test (‘this seems useful’) but failed a rigour test (‘here’s the brief, here’s the owner, here’s the count, here’s the kill criterion’). The checklist exists because the emotional test isn’t enough. AI work is honest — it will execute exactly what you brief, exactly the way you brief it. Which means the only way to fail is upstream of the code.

_{THE PATTERN}

None of the mistakes were technical.

Read all five again. Bad scope. Bad debugging discipline. Bad ownership. Bad stack hygiene. Bad measurement. Every single one is a thinking mistake, not a technical mistake.

That’s the pattern. The technical layer is honest. It will execute exactly what you ask, exactly as you wrote it. Which means the only place mistakes can come from is upstream — from the decisions about what to build, who owns it, how often it runs, and whether it’s worth building.

Both terrible and great news. Terrible because you can’t blame the AI. Great because thinking is improvable, and the feedback loop in AI work is days, not quarters. Every mistake on this list made me a better operator faster than the equivalent mistake in a non-AI medium would have.

THE REFRAME
If you’re trying to do this work, expect to make these mistakes. They’re not signals you’re bad at this. They’re tuition. The cost of getting better at AI work is paid in mistakes that show up upstream of the code. The reward is that you become a better operator, not just a better AI user. That’s the part the wins-only content doesn’t mention.

If this was useful

This is part 4 of a 5-part editorial series on building AI workflows that actually compound. Real receipts from systems we run at Know AI. No jargon, no hype.

Follow @zeeel.ai on Instagram for the rest of the series.

TELL ME YOURS
Send me a mistake from your own AI work. I’ll trade you a question I’d ask, in DMs.
Keyword: MISTAKES. Patterns across the replies become a future post — with permission, and never naming individuals.