June 22, 2026

How to measure ROI on an AI agent before you build it (and after)

You've seen the demos. The pitch sounds great: "Cut support costs by 60%, respond in seconds, scale without hiring." But you're the one writing the check, and "AI agent" projects have a reputation for landing somewhere between magical and money-pit. Before you sign anything, you need a way to size the prize — and after launch, you need to know whether it actually paid off.

Here's the financial framework I walk clients through before we build anything.

The pre-build math: size the deflection opportunity

Most AI agent pitches skip the part where you figure out if the project is worth doing. Don't skip it. The math is simple:

Monthly ticket volume × average handle time × fully-loaded hourly cost = your support spend

Let's run a realistic example for a 25-person business doing e-commerce or SaaS:

800 support tickets per month
12 minutes average handle time (including context switching, lookup, response)
$35/hour fully-loaded cost for your support rep (salary + benefits + tools + overhead)

That's 800 × (12/60) × $35 = $5,600/month in human support cost, or about $67K/year.

Now ask: what percentage of those tickets are repetitive — "where's my order," "how do I reset my password," "do you ship to Canada," "what's your return policy"? If you've never measured it, take a sample of 100 tickets and categorize them. In my experience the answer is usually somewhere between 40% and 70%.

If 50% of your tickets are deflectable and an agent handles them well, you're looking at potentially $2,800/month in reclaimed capacity. That's not money you put back in your pocket — it's hours your team gets back for higher-value work (or hires you don't have to make as you grow).

Set your break-even number now, in writing. Something like: "This agent needs to deflect at least 30% of tier-1 tickets within 90 days or we pull the plug." If the vendor flinches at that, you've learned something useful.

The 3 metrics that actually matter

Once the agent is live, these are the only numbers I'd put on a dashboard:

1. Deflection rate

The percentage of incoming conversations the agent fully resolves without a human ever touching it. "Fully resolves" means the customer didn't immediately re-open the ticket, didn't escalate, and didn't come back with the same question 48 hours later.

A healthy deflection rate for a well-scoped tier-1 agent is 25–50% in the first 90 days. If a vendor promises 80% out of the gate, ask them what they're counting.

2. Escalation rate (and escalation quality)

What percentage of conversations get handed off to a human, and when that happens, does the human get useful context? A good agent fails gracefully — it captures the customer's question, what it tried, and any account details, then passes a clean handoff. A bad agent dumps an angry customer into a blank chat window with your overworked rep.

Track: % escalated, average time-to-escalation, and rep satisfaction with the handoff (ask them, monthly).

3. CSAT split: agent-handled vs human-handled

This is the one most teams skip and it's the most important. Send a one-question CSAT survey after every resolved ticket and tag whether the agent handled it solo. If your agent-handled CSAT is within 10 points of your human CSAT, you're winning. If it's 30 points lower, your customers are tolerating the bot, not appreciating it — and that's a churn risk you're not seeing on the cost-savings spreadsheet.

The 3 metrics that don't matter (no matter what your dashboard says)

"Total conversations handled"

Vanity metric. An agent that "handled" 5,000 conversations by giving useless answers to 5,000 people is not a win. This number gets quoted in vendor case studies because it's big and sounds impressive.

Response time in isolation

Yes, fast responses are good. But a 2-second wrong answer is worse than a 2-minute right one. Response time only matters paired with resolution quality.

"AI resolved" without a quality audit

Most agent platforms let the agent self-report whether a ticket was "resolved." This is like asking a student to grade their own test. You need a human spot-check — pull 20 random "AI resolved" conversations per week and actually read them. Tag the ones where the customer got bad info or a runaround. That's your real resolution rate.

A 90-day payback example

Let's run actual numbers. Say a build comes in at $3,000 one-time plus $500/month in platform, model, and maintenance costs. That's $4,500 total over 90 days.

Using our earlier example — 800 tickets/month at $5,600/month in support cost — what deflection rate breaks even in 90 days?

$4,500 ÷ 3 months = $1,500/month it needs to save
$1,500 ÷ $5,600 monthly support cost = 27% deflection rate

So this project pays for itself within the first quarter if the agent handles roughly 1 in 4 tickets cleanly. After month 3, the monthly cost drops to just the $500 run rate, and at 30% deflection you're netting around $1,180/month in reclaimed capacity — call it $14K/year on a $3K initial investment.

Two things to notice:

The break-even bar is lower than most vendors imply. You don't need an 80% deflection rate to justify the spend. You need a realistic 25–35%.
The math collapses fast if you skip the audit step. If your "deflection rate" includes tickets where the customer gave up and went to a competitor, you're not saving money — you're paying for churn.

When ROI doesn't materialize (and why)

I'd rather tell you when this fails than oversell when it works. Three patterns I see:

Query complexity is too high. If 70% of your tickets require account-specific context, multi-step troubleshooting, or judgment calls, an agent isn't the right first investment. Fix your help docs and self-service flows first. An agent without a clear tier-1 lane to operate in will just frustrate everyone.

Your knowledge base is stale. The agent only knows what you feed it. If your help docs reference a product version from 2022, or your policies live in a Slack thread, the agent will confidently give wrong answers. Plan to spend the first two weeks of any build cleaning up source content. This isn't glamorous and no vendor likes talking about it, but it's where most projects die.

No feedback loop after launch. The agent that goes live on day one is not the agent you want running 6 months later. You need a weekly habit: review escalations, find the patterns, update the knowledge base or the agent's instructions. Without that, performance decays. Most "AI failed us" stories I hear are really "we launched it and never touched it again" stories.

What this looks like in practice

When I scope an agent build, the first deliverable isn't a chatbot — it's the math above, filled in with your real numbers. We pull a sample of your actual tickets, categorize them, estimate a realistic deflection range, and decide together whether the project clears the bar.

If the numbers don't work, I'll tell you. Sometimes the right answer is "spend $800 on better help docs and a smarter contact form first, then revisit the agent in 6 months." That's a worse outcome for me in the short term and a much better one for you.

If the numbers do work, we scope a pilot — narrow lane, defined success metrics, 90-day evaluation window. After 90 days you have hard data on deflection rate, escalation rate, and CSAT, and you decide whether to expand, hold, or pull the plug. No 12-month contracts. No "trust the process."

Run the numbers before you sign anything

The AI agent market is full of vendors who'll happily build you something impressive that doesn't move a single business metric. The pre-build math protects you from that. The post-launch metrics keep you honest. And the willingness to kill a project that isn't working is what separates a real ROI conversation from a sunk-cost spiral.

If you want help running these numbers for your business — actual ticket data, actual deflection estimate, actual go/no-go recommendation — that's the discovery call I offer before any AI Pilot engagement. No pitch deck, just math. You can book a call or send a note through thewizrdz.io or learn more about the AI Pilot package.

Build the agent if the math says build it. Don't if it doesn't. That's the whole framework.