AI ROI: Only 13% of CIOs See It, and Stopping Rules Explain Why

Gartner published a number on October 19 that should change how you plan your next AI project. In its survey of 2,002 CIOs and technology executives, only 13% said their enterprise has realized significant or transformational value from AI. The odds of any single use case succeeding are still about one in five. If your board is asking about AI ROI, that's the backdrop.

The more useful part of the release is what separates the winners. Gartner's "vanguard" CIOs aren't running more pilots. They set stopping rules: clear criteria for when a pilot should not move to production, and when a system already in production should be switched off. Very few companies have either.

TL;DR

• Companies with written rules for stopping pilots are 2.3 times more likely to be AI leaders, Gartner says. Only 31% of CIOs have them.
• AI ROI depends on deciding upfront what "not worth it" looks like, including run-time costs.
• Put kill criteria in the scope and contract before any code is written.

What Gartner found about AI ROI

The data comes from Gartner's 2027 CIO and Technology Executive Agenda survey, fielded between May and July 2026. The detail matters more than the headline:

• Organizations with guidelines for deciding when a pilot should not go to production are 2.3 times more likely to be vanguards. Only 31% of CIOs have such guidelines.
• Organizations with guidelines for abandoning low-value AI that's already live are 2.6 times more likely to be vanguards. Only 17% have them.
• 58% of CIOs report more pressure to deliver cost improvements from AI, and 51% expect AI to raise the total cost of ownership of their technology.

Alicia Mullery, VP Analyst at Gartner, put it bluntly: "Volume is not value."

Other research agrees. MIT NANDA's 2025 study, reported by Fortune, found that about 95% of generative AI pilots produced no rapid revenue impact. And Gartner predicted in 2025 that over 40% of agentic AI projects will be canceled by the end of 2027 because of rising costs, unclear value or weak risk controls. Projects get canceled anyway. Better to cancel early and cheaply than late and expensively.

Why stopping rules matter more than another AI pilot program

Most AI pilot programs are set up to prove something can work. Almost none are set up to prove it's worth running. The second test is harder.

A pilot that answers 80% of support tickets correctly on a curated test set has proven feasibility. It hasn't proven that agents will trust it or that full-volume inference costs beat today's cost per ticket. Without a pre-agreed bar, teams drift into "one more sprint" mode.

Gartner also notes that AI costs are probabilistic and set at run time. A SaaS licence costs the same in month 12 as in month 1. An LLM application's bill moves with traffic, prompt length, retries and model choice. It's a big reason why AI projects fail: the problem is often cost, not accuracy. The pilot looked cheap because it ran on 500 requests a day.

Two gates, not one

The vanguards set rules at two points:

1. Pilot to production. Does the system meet a measured bar on quality, cost and adoption, against a real baseline?
2. Production to retirement. Is the live system still earning its keep six or twelve months later?

The trade-offs: who owns the AI ROI decision

Stopping rules only work if someone can enforce them. That's partly a team-model question.

Each model has its own incentive problem. An internal team may not want to kill its own project. An external provider is paid to keep going. That's not a reason to avoid outside help. MIT's research found that solutions bought from specialized vendors or built with partners succeeded about 67% of the time, versus roughly a third as often for internal builds. It's a reason to write the stopping rule down before anyone has a stake in the answer.

More in our guide to in-house vs FDE vs staff augmentation.

What this means for your AI proof of concept

If you're scoping an AI proof of concept next quarter, three things change.

Scope gets a "no" section. Alongside goals and deliverables, write down the result that ends the project. Example: "If the agent resolves fewer than 40% of tier-1 tickets without escalation by week 8, we stop." The number is yours. It just has to exist before week 1.

Budget includes run time, not just build. Model the cost per transaction at expected production volume, not pilot volume. Then set a ceiling. If cost per resolved ticket beats your current cost per ticket only at pilot volume, the case doesn't hold.

Timeline gets checkpoints with teeth. A 12-week engagement might look like this:

• Weeks 1–2: baseline the current process and agree the eval set.
• Weeks 3–6: build and run evals weekly. Week 6 is a soft gate.
• Weeks 7–10: limited production traffic, measuring adoption and cost per transaction.
• Week 11: go or stop decision against the written criteria.
• Week 12: handover or wind-down, documented either way.

Our guide on how to scope your first AI deployment goes deeper on the baseline and eval steps.

How to write AI kill criteria: a checklist

Adapt these stopping rules and agree them with the business owner, not just engineering. Gartner calls this "a team sport" for a reason.

Gate 1: pilot to production

1. Quality: the system hits an agreed score on a held-out eval set built from real cases, not vendor demos.
2. Baseline: it beats the current process on at least one measured business metric (cost, cycle time, error rate).
3. Run-time cost: projected cost per transaction at full volume is below a stated ceiling.
4. Adoption: a defined share of pilot users choose to use it in week 8, not just in week 1.
5. Risk: security, data access and failure handling are signed off.

Gate 2: keep running or retire

1. Review every quarter against the same metrics, with a named owner.
2. Retire or rework if value falls below the threshold for two quarters in a row.
3. Track cost drift: model changes, traffic growth and prompt changes all move the bill.

To sanity-check the money side before you commit, run your build and run-time assumptions through the free AI deployment cost calculator.

Contract clauses to ask an external team for: milestone payments tied to gate decisions, a clean exit at each gate without penalty, ownership of code and eval sets, and handover documentation regardless of outcome. See also planning an exit from an FDE engagement.

Find the right deployment partner

Good providers welcome kill criteria, because clear gates protect their reputation too. Ask teams how they set baselines and what they hand over if a project stops. Provider matching compares teams against your requirements with plain-language reasons and gaps, and the directory shows verification status on every profile and labels any featured placement. If you're still deciding whether to build or bring in help, start with the AI Deployment Assessment. Find an FDE provider.

Key takeaways

• Gartner found that only 13% of CIOs report significant AI value, and one in five use cases succeeds.
• The leaders write stopping rules at two gates: pilot to production, and production to retirement.
• AI ROI has to include run-time cost at production volume.
• Put kill criteria in the scope and the contract before anyone has a stake in continuing.
• External teams can raise your odds, as long as gate decisions stay with you.

FAQ

How do you measure AI ROI?

Start with a baseline of the current process: cost per transaction, cycle time and error rate. Then measure the AI system against that baseline in limited production, including run-time costs at expected volume. ROI is the improvement in those business metrics minus build, run and maintenance costs. Measure it again every quarter, because usage and model costs change over time.

Why do AI projects fail to reach production?

The common causes are unclear business value, costs that grow faster than expected and weak risk controls, which Gartner cites for agentic AI cancellations. Many pilots also prove feasibility on curated data without testing adoption or full-volume cost. Without agreed criteria upfront, projects drift for months before anyone decides to stop them.

What should an AI proof of concept include?

A good AI proof of concept includes a measured baseline, an eval set built from real cases, a target result, a cost ceiling at production volume and a written stopping rule. It should also name the business owner who makes the go or stop decision, and set out what gets handed over if the project ends.

When should you stop an AI pilot program?

Stop when the pilot misses the criteria you agreed before it started: quality below the bar, projected run-time cost above your ceiling, or low adoption by real users after the novelty fades. If you never set criteria, set them now and give the pilot one fixed checkpoint to meet them.

Before your next pilot starts, write one sentence that says what result would end it, and get the business owner to sign it. Then work out if you should build, hire or partner. Take the free AI Deployment Assessment.

Sources

• Gartner Survey Finds Only 13% of CIOs Say Their Enterprise Has Realized Significant or Transformational Value from AI (Gartner, October 19, 2026)
• Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027 (Gartner, June 25, 2025)
• MIT report: 95% of generative AI pilots at companies are failing (Fortune, August 18, 2025)