AI Project ROI: What to Actually Measure

A CFO at a mid-market insurance firm pulled up a slide last quarter. The headline number: 340% ROI on their AI customer service deployment after six months. Impressive. I asked him to walk me through the math. Twenty minutes later, we had a real number. It was not 340%. It was closer to 61%.

Nobody lied. The finance team built the model in good faith. They just measured the wrong things, ignored the costs they could not see, and counted benefits that never touched the bottom line. This is the default state of AI project ROI in almost every company I walk into. And it is the single biggest reason why leaders either keep pouring money into initiatives that do not work or kill initiatives that actually do.

Why Most AI ROI Numbers Are Fiction

When a vendor or a pilot team shows you a big ROI number, it usually rests on three shaky pillars. I have seen this pattern across dozens of AI strategy engagements.

Pillar one: fake time savings. Someone estimates that a workflow used to take 40 minutes and now takes 10. That is a 30-minute gain, multiplied by the number of times per week, multiplied by the hourly rate. Clean math. Except the original 40-minute number came from a gut estimate, not a measurement. And the 10-minute number assumes the human is fully productive with the freed-up time, which almost never happens without deliberate redeployment.

Pillar two: missing costs. Tool subscription? In the model. Integration engineering? Almost never. Training time? Rarely. The productivity dip during adoption when your team is half on the new system and half on the old one? Never. These costs are real, and they are often 40 to 60 percent of the total investment over a twelve-month window.

Pillar three: vanity metrics masquerading as value. "Customer response time dropped 70%" sounds great. But if nothing measurable changed for customers, retention, or revenue, that number is decoration. Not value.

The result is that leadership either commits to initiatives that never pay back, or kills honest initiatives that are quietly delivering 30 to 80 percent real returns because the reported number could not be defended under scrutiny. Both outcomes are avoidable with a better framework.

What Counts as Real Value in an AI Project?

Here is the honest answer: only three things should go in the value column of your AI project ROI model. Anything else is soft and needs a skeptic’s eye before it earns a dollar figure.

• Hours saved, converted to dollars using fully loaded labor cost. Fully loaded means salary plus benefits plus overhead, usually 1.3 to 1.4 times base salary for most companies. And the hours must be measured, not estimated. More on that in a minute.
• Errors prevented, converted to dollars using real cost per error. If an AI system catches compliance mistakes that cost $3,000 each to fix after the fact and prevents 40 per year, that is $120,000 in hard value. This number is defensible because both inputs are observable.
• Revenue from faster delivery cycles. If your proposal turnaround goes from five days to two and that measurably increases your win rate, the delta in closed revenue is real. But you need the attribution data to prove it, not just a vibe.

That is the whole value side. Not "improved employee satisfaction." Not "better customer experience." Not "future-proofing." Those things matter, but they do not go in the ROI model. They go in a separate section called strategic benefits, and leadership weighs them alongside the hard number.

The Hidden Costs That Destroy Paper ROI

Here is where most AI project ROI models quietly fall apart. The vendor sends you a template. The template asks about tool cost and expected time savings. It does not ask about any of these:

• Integration engineering. Wiring the AI into your existing systems almost always costs more than the tool subscription. A $15,000 per year tool often needs $40,000 to $80,000 of engineering work before it produces anything.
• Training and change management. Your team has to learn the new workflow. Someone has to write the playbook. Someone has to handle the resistance. Budget real hours here, not a token four-hour workshop.
• Oversight time. For the first three to six months, someone experienced has to review AI output to catch errors and calibrate the system. That is real labor, and it comes out of the productive hours of your best people.
• The adoption dip. When you roll out a new workflow, throughput drops before it rises. Plan for a 15 to 30 percent productivity hit for the first four to eight weeks. If you do not account for this, your first-quarter ROI report will look terrible and leadership will panic.
• Maintenance and drift. Models need retuning. Prompts break when the vendor updates. Data pipelines shift. You need someone on the hook to maintain the system, and that is an ongoing cost, not a one-time line item.

I use a rough rule with clients: if your initial ROI model does not show these five cost buckets explicitly, the number is wrong by at least 30 percent. Usually more. Anyone who tells you otherwise is selling something.

How Do You Actually Measure Time Saved?

This is where most AI project ROI claims completely fall apart. The honest way to measure time saved is to run a baseline, a midline, and an endline. Skipping any of them means your "time saved" is an anecdote, not a measurement.

Baseline (week zero). Before you deploy anything, pick five to ten people and track how long the target workflow actually takes. Have them log it for a full week. The results will surprise you. The number your team estimates is usually off by 20 to 50 percent in either direction.

Midline (weeks four to six). This is during active adoption. Track again. Expect worse numbers than baseline. That is normal. If you report ROI at this stage, you will look bad. Do not report yet.

Endline (weeks ten to twelve). After the team has fully adopted the new workflow and the dip has resolved, measure again. The delta between baseline and endline, multiplied by frequency and fully loaded cost, is your real time savings number.

I worked with a professional services firm that used this exact approach for their AI proposal-generation rollout. Baseline: 11 hours per proposal. Midline at week five: 13 hours. Endline at week twelve: 4.5 hours. Real savings: 6.5 hours per proposal. At their loaded cost and proposal volume, that worked out to roughly $180,000 per year in recovered capacity. Defensible, measurable, and useful for planning.

The 90-Day Measurement Cadence

Here is the cadence I walk clients through when we set up AI project ROI measurement. It is the same every time because it works.

• Week 0: Establish baseline. Log current time, error rates, throughput. Document what "good" looks like before anything changes. Get finance to sign off on the baseline methodology so there are no arguments later.
• Week 4: First checkpoint. Expect worse numbers than baseline. Identify adoption friction. Adjust training and support. Do not report ROI externally. This is a learning week, not a judgment week.
• Week 8: Second checkpoint. Numbers should be approaching baseline or slightly above. Refine the workflow. Identify edge cases. Lock down the scope of what the AI handles and what escalates to humans.
• Week 12: First real ROI calculation. Measure against baseline. Subtract all the hidden costs we discussed. Present the honest number to leadership, with a clear explanation of methodology.
• Months 4 through 12: Quarterly review. The real long-term ROI often diverges from the 90-day number, usually in a positive direction as the team gets better at using the tool and expands it to adjacent workflows.

This cadence prevents two common failure modes. It stops teams from killing initiatives at week five when things look bad because the adoption dip is still underway. And it prevents teams from celebrating at week four with numbers that have not yet survived scrutiny.

The Case Study That Changes How You Think About AI ROI

Back to the insurance firm with the 340% number. Here is what the real model looked like after we rebuilt it together.

The original calculation counted all agent time saved at full billable rate, assumed zero hidden costs, and credited the AI for improvements in customer satisfaction scores that were actually driven by a separate coaching initiative. When we ran it through the framework above, the real numbers were:

• Gross value: $412,000 per year. Solid time savings and measurable error reduction in claims routing.
• Real costs: $256,000 over 12 months. Tool, integration, oversight, and adoption dip.
• Real 12-month ROI: 61%. Net benefit of $156,000 on a $256,000 investment.

The CFO was initially disappointed. I told him 61% was excellent. It was real, defensible, and most importantly it was a floor, not a ceiling. By month 18, as the team expanded the system to adjacent use cases and the adoption curve settled, they hit 140% ROI on the same base investment. That second number would not have been possible if they had killed the initiative at month three based on the honest numbers. The discipline of measuring correctly saved the project.

The lesson I tell every leader I work with: honest ROI beats inflated ROI every time. Inflated numbers die on contact with reality. Honest numbers compound.

What to Do This Week

If you have an AI initiative running right now and you cannot confidently answer these questions, you are flying blind:

• What was the measured baseline before we deployed this?
• What are our real hidden costs, line by line?
• What happens to the freed-up human capacity? Is it actually being redeployed to higher-value work?
• When is our next formal ROI review, and what methodology will we use?

Pick your biggest AI initiative and answer those four questions this week. If you cannot, rebuild your measurement framework before you rebuild anything else. We cover the related challenge of picking the right workflows in AI workflow automation: stop automating the wrong things, which pairs naturally with this piece.

If you want an outside read on your AI project ROI model, or help building one from scratch. That is exactly what we do. Explore our AI strategy services or reach out. We will tell you the honest number. That is usually worth more than the inflated one.