AI Replaced Someone On Our Team
F I E L D N O T E · T E A M D E S I G N
AI didn't replace the role. It replaced the tasks.
Six months after handing a role on our team to a workflow, here's the honest version of what changed, the stage where it broke, and the reframe that finally made it work.
6 mo
Since the swap
2 wks
Of false signal
20%
Still owned by a human
The role we replaced wasn't really the role.
We replaced a role on our team with AI. We weren't sure it was the right call. It half-was and the half that wasn't cost us more than we'd planned to save.
The work looked like a textbook automation candidate. Research. Summarising. First drafts. Formatting reports. Good work but the same work, every single week. So we built a workflow to handle it, and told ourselves it would be fine.
What we missed is the part of the job that doesn't show up on the spec sheet. A person doing repetitive work isn't only running a process. They're noticing. They flag the number that doesn't match the previous four. They pause when a brief feels thin. They say, out loud, "this doesn't seem right." None of that was in the workflow we built.
The reframe: AI doesn't replace a role. It replaces the tasks within a role. The judgment, the gut feel, the "wait, something's off" feeling still needs a human. Just one doing far less of the repetitive stuff.
Below is the actual journey, broken into the five stages we lived through. If you're about to do the same thing, this is the version of the story we wish someone had handed us in week zero.
STAGE 01 · PRE-LAUNCH
The Trap
Confusing "this repeats every week" with "this is safe to automate."
The role was repetitive but important. Research, summarising, first drafts, formatting reports. Same shape every week. That repetition is the trap. Repetition reads as a signal that the work is mechanical. It almost never is.
A person doing the same task every week isn't just running a process. They're building a baseline. The fifth time they see a number, they flag the one that doesn't match the previous four. The tenth time they read a brief, they notice when one is thinner than usual. That noticing isn't on the job description.
When we wrote the spec for the workflow, we copied the visible half of the job. We didn't copy the noticing, because we couldn't see it.
WHAT WE WROTE DOWN · THE WORKFLOW SPEC
A clean, repeatable spec. Inputs go in, formatted output comes out. Looks complete. Isn't.
WHAT LOOKED SAFE
- Inputs were structured and consistent.
- Output had a fixed template.
- Steps were the same every week.
- The role had clear, measurable deliverables.
WHAT WE COULDN'T SEE
- The person also flagged anomalies in the inputs.
- They asked the source when a brief looked thin.
- They paused on numbers that didn't match the trend.
- They escalated when something felt wrong.
The mistake at this stage: treating "repeats every week" as a proxy for "safe to automate." Repetition is what made the human good at this work. It's not what made the work mechanical.
STAGE 02 · WEEKS 1-2
The Honeymoon
Two weeks of speed that felt like proof.
The first two weeks felt like a win. Output was faster. Quality was close enough. Nobody had to chase anyone for a deliverable. Every metric we'd defined for success came back green.
What we didn't have was a metric for "did anything go wrong that nobody mentioned?" Our dashboard tracked throughput, turnaround, and format compliance. It didn't track the absence of things, the unflagged anomaly, the un-asked clarifying question, the un-escalated weird input.
The first two weeks of any automation will feel like a win. That's because the failures are silent and you haven't found them yet.
WHAT OUR DASHBOARD SHOWED · WEEK 2
WHAT WE MEASURED
- Speed of delivery.
- Format and template compliance.
- Cost per output.
- Number of escalations (zero - read as good).
WHAT WE SHOULD HAVE MEASURED
- Rate of self-flagged uncertainty in outputs.
- Sample-review accuracy on a random pull.
- Anomalies caught in the inputs (now zero).
- Questions asked back to the source.
The mistake at this stage: assuming green metrics meant the work was correct. Speed is a lagging indicator. Silence on a dashboard is not the same as everything being fine.
STAGE 03 · THE DRIFT
The Silence
Work completing perfectly. Silently. Wrong.
Then something started showing up that we didn't expect. Reports that read fine on the surface but pointed at the wrong number. Summaries that confidently described a brief that wasn't quite the brief we'd been sent. Drafts that smoothed over an inconsistency a human would have flagged within the first paragraph.
When a person does something, they flag problems. They notice when something feels off. They say, in a Slack message or in a comment, "this doesn't seem right." The AI just completed the task. Perfectly. Silently. Wrong.
A flagged error is half-fixed before anyone touches it. Someone knows it exists. Someone is watching it. A silent error compounds, it ships, it gets cited downstream, and the next person assumes it was reviewed.
WHAT WAS ACTUALLY HAPPENING · WEEK 5
WHAT HUMANS DO AUTOMATICALLY
- Pause on inputs that feel weird.
- Ask back when a brief looks thin.
- Re-check work when a number doesn't match the trend.
- Say "this doesn't seem right" out loud.
WHAT AI DID INSTEAD
- Completed every task on time.
- Produced fully formatted output every run.
- Filled gaps in thin briefs without flagging them.
- Returned zero questions and zero escalations.
The mistake at this stage: mistaking "no flags raised" for "no problems." A junior with zero questions in their first month isn't a star — they're a risk. Same rule applies here.
STAGE 04 · THE AUDIT
The Reckoning
We caught it eventually. But not for free.
We caught it eventually. We always do. But the cost wasn't the wrong outputs. Wrong outputs are fixable. The cost was the time we'd thought we were saving, returned to us with interest. Hours redoing reports. Hours auditing back through the chain to figure out what else might be off. Hours rebuilding trust with the people who'd received the wrong work.
The lesson wasn't that AI failed. The AI did exactly what we'd asked it to do. The lesson was that nobody was watching the way you watch a person. A junior produces work and gets reviewed. An automation produces work and gets shipped. We had set up a hire. We hadn't set up a review loop.
THE HIDDEN LEDGER · 6-MONTH AUDIT
Two columns we should have run from week one. Hours saved on paper versus hours actually saved after the audit cleared.
COST ON THE SPREADSHEET
- Salary saved on the role.
- Throughput up across the team.
- Faster turnaround on every report.
- Fewer chase emails per week.
COST WE DIDN'T TRACK
- Rework hours against silently wrong outputs.
- Audit hours rebuilding the trail of decisions.
- Trust capital spent with people who got bad work.
- Slower decision-making while we were unsure.
The mistake at this stage: not reviewing AI work the way you'd review a junior's. If you'd watch a new hire's first month closely, watch the automation's first six. The trust budget is similar. The review loop should be too.
STAGE 05 · THE AUDIT
The Rebuild
Smaller team. Same accountability. New shape.
What we have now isn't what we planned. Smaller team. Faster output. But someone still owns the quality, they just spend their time on the 20% that actually needs a brain. The judgment, the gut feel, the "wait, something's off." That's the part we couldn't see when we wrote the spec, and it's the part that survived.
The role didn't disappear. It changed shape. Where the old role was 80% production and 20% noticing, the new role is the inverse:
20% production (review, polish, edge cases) and 80% noticing — across more output than one person used to handle.
The economics still work. Just not in the way we'd modeled them. We didn't replace headcount. We rebalanced what humans spend their hours on.
THE NEW SHAPE · HOW THE ROLE REBUILT ITSELF
The same role, restructured. AI runs the production tasks. A human owns the noticing. Output is higher because the human's hours moved up the value stack.
HAND TO AI
- Drafting at scale, mechanical research.
- Summarising, formatting, templating.
- Anything where the input is structured.
- Anything you'd describe as "the same shape every time."
KEEP HUMAN
- Final review on every output before it ships.
- Anomaly noticing in the inputs.
- Anything that touches a customer, a number, or a decision.
- Edge cases — by definition, off the spec sheet.
The reframe at this stage: AI doesn't replace a role. It replaces the tasks within a role. Build for that, and the math actually works.
Before you hand a role to AI, ask these.
Five questions we wish we'd answered in week zero. They take an afternoon. They would have saved us six months.
Q1 — Spec
Is this work actually mechanical, or does it just repeat? Repetition often means a human has built up a baseline you can't see.
Q2 — Noticing
Where does the current human flag, ask, escalate, or pause? Write those down. They are the part of the role that isn't safe to automate.
Q3 — Review loop
Who is reviewing the AI's output before it ships, the way you'd review a junior's? If the answer is "nobody," you don't have an automation, you have a liability.
Q4 — Silent-wrong check
What's the signal that something is wrong but unflagged? Sample reviews, anomaly alerts, or a comparison against last period. Without one, every dashboard lies green.
Q5 — Owner of the 20%
Who owns the 20% that still needs a human after the rest is automated? Name them. Pay them. Give them the time. That role is the whole point.
How to do this without paying our tuition.
Don't start with the role. Start with the tasks inside it. List every task the role does in a normal week. Mark each one as production (drafting, formatting, summarising, mechanical research) or judgment (noticing, escalating, deciding, owning the outcome).
Hand the production tasks to AI. Keep the judgment tasks with a person. Then build a review loop on top of the production tasks for at least the first quarter, sample-review every output, the way you'd review a new hire's. Track the absence of flags as carefully as you track throughput.
If the metrics still work after the review loop, you've found a real win. If they don't, you've discovered something more useful: which parts of the role you actually couldn't see.
AI will change your team. Probably not in the way you're imagining right now. The teams that get this right are the ones who figure out what to keep human, not just what to automate.
AI didn't replace the role. It replaced the tasks. · A field note