AI Pilot Purgatory: Why Companies Struggle to Scale AI

AI pilot purgatory is when a company keeps testing AI in pilots and proofs of concept that never become part of everyday operations. It's widespread: in a Gartner survey published in September 2026, only 22% of organisations had scaled AI across multiple business units. The causes are rarely the model itself. They're integration, data, governance, process and people.

This guide sets out what the data really says about pilots that stall, why they stall, what companies that scale do differently, how the new wave of AI agents fits in, and what it means for UAE businesses and real estate firms.

Key takeaways

• There's no single failure rate. Depending on the source and definition, between about 5% and 44% of organisations report scaling AI or getting value from it at scale. Quote a range, not one number.
• The viral "95%" figure is weaker than it sounds. MIT NANDA's 2025 report rests on 52 interviews and 153 survey responses; by its own figures, about one in four organisations that piloted custom AI tools got them into production.
• Workflow redesign is the strongest signal. In McKinsey's 2026 survey, nearly three-quarters of AI high performers had fundamentally redesigned workflows, against a quarter of other respondents.
• Agents are the next pilot wave. Gartner expects more than 40% of agentic AI projects to be cancelled by the end of 2027, and only 21% of the companies planning agents in Deloitte's survey have a mature governance model for them.
• UAE adoption is high; readiness to scale is not. Microsoft ranks the UAE first for AI use, but in a Korn Ferry survey of 105 Gulf organisations only 1% considered themselves fully AI-ready.

What is AI pilot purgatory?

AI pilot purgatory is a state of permanent experimentation: pilots that look promising in a demo but never become the way work is actually done. The company keeps funding proofs of concept, while the processes, systems and people around them stay the same.

Definition

AI pilot purgatory — the gap between testing AI and running it at scale, where pilots or proofs of concept keep going, stall or quietly end without being integrated into operations, and without producing measurable business results.

Analysts measure it in different ways, which is why the headline numbers vary so much. Gartner counts projects abandoned after the proof-of-concept stage, IDC counts how many proofs of concept reach production, and McKinsey and Deloitte ask executives how far they've scaled. None of them is wrong, but they aren't interchangeable.

How common is AI pilot purgatory?

Very common, although the size of the gap depends on who's counting and how.

Source	Who was asked	What it found	Caveat
Gartner (September 2026)	1,303 organisations with revenue of $50 million or more, January–April 2026	Only 22% have scaled AI across multiple business units	Self-reported
McKinsey (August 2026)	1,719 participants in 97 countries, May–June 2026	44% say AI is scaling across their enterprise; 37% say AI has contributed to EBIT	Respondent-level, self-reported
Deloitte (January 2026)	3,235 business and IT leaders in 24 countries	Only 25% have moved 40% or more of their AI pilots into production	Leaders already using AI
BCG (September 2025)	More than 1,250 companies	5% achieve AI value at scale; 60% lag in building the capabilities	Self-assessed maturity
S&P Global (2025)	1,006 IT and business professionals in North America and Europe	42% abandoned most AI initiatives before production, up from 17%; 46% of projects scrapped on average	Survey of mid-level and senior staff
IDC for Lenovo (2024 data)	Organisations in Asia-Pacific	For every 23 AI proofs of concept, 3 reached production	Vendor-commissioned

Gartner's own story shows how the problem grew. In July 2024 it predicted that at least 30% of generative AI projects would be abandoned after proof of concept by the end of 2025, because of poor data quality, inadequate risk controls, escalating costs or unclear business value. In January 2026 a Gartner analyst wrote that at least 50% had been, although the article doesn't publish a method or sample.

Common misconception

"MIT proved that 95% of AI pilots fail." MIT NANDA's July 2025 report said 95% of organisations were getting zero return from generative AI, but it was a preliminary study based on 52 interviews, 153 survey responses from conference attendees and a review of public initiatives, with return measured about six months after a pilot. Its own funnel for custom AI tools ran from 60% of organisations evaluating, to 20% piloting, to 5% in production: roughly one in four piloting organisations made it. Fortune's widely shared write-up also overstated the sample. The study is a warning about weak pilots, not a failure rate. For how to measure return properly, see enterprise AI ROI.

Why do AI pilots stall?

Rarely because the model can't do the task. MIT NANDA found that what holds most AI tools back is that they don't learn and don't integrate well into workflows. The evidence points to six recurring causes.

• Integration. A pilot can run on its own; production has to connect to the CRM, ERP, document systems and security controls. In Korn Ferry's 2026 Gulf survey, technology integration was the biggest barrier to AI return (61%). Gartner warns that fitting agents into legacy systems can require costly changes.
• Data. In a Gartner survey of 1,203 data management leaders, 63% said their organisations either lacked or weren't sure they had the right data management practices for AI. Gartner predicts that through 2026, organisations will abandon 60% of AI projects that aren't supported by AI-ready data.
• Governance and risk. Inadequate risk controls appear in both of Gartner's abandonment forecasts. Organisations that regularly audit and assess their AI systems are over three times more likely to get high value from generative AI, according to a Gartner survey of 360 organisations.
• Process. McKinsey found that of 25 attributes tested, redesigning workflows had the biggest effect on whether companies saw profit impact from generative AI, yet only 21% of respondents using generative AI said their organisations had fundamentally redesigned at least some workflows. In Deloitte's 2026 survey, 30% were redesigning key processes around AI and 37% were using it only at a surface level. The how-to is in AI workflow redesign.
• People. Deloitte's respondents saw the AI skills gap as the biggest barrier to integration. BCG's AI leaders follow a rule of putting 10% of resources into algorithms, 20% into technology and data and 70% into people and processes.
• Cost and measurement. Gartner put the cost of generative AI deployments at $5 million to $20 million depending on the approach, and in a May 2025 survey 72% of CIOs said their organisations were breaking even or losing money on AI. Only 14% of CEOs in BCG's 2026 study clearly define the P&L impact for all their AI initiatives, and McKinsey found fewer than one in five organisations tracking KPIs for generative AI.

From pilot to production

1. 01Choose
   • A business owner for the process
   • A baseline and a target KPI
   • A decision date

   Before anything is built

2. 02Pilot for production
   • Real data and real users
   • An agreed integration path
   • Security and risk review

   Test what you'll actually run

3. 03Decide
   • Scale, fix or stop
   • Compare with the baseline
   • Write down the reasons

   On the date, not later

4. 04Scale
   • Redesign the workflow
   • Train the people
   • Monitor cost and quality

   Where the value shows up

What do companies that scale AI do differently?

The research points in a consistent direction.

1. They redesign the work, not just add a tool. In McKinsey's 2026 survey, nearly three-quarters of AI high performers had fundamentally redesigned workflows, up from 55% a year earlier, against a quarter of other respondents. BCG found high performers roughly seven times more likely to redesign workflows and reshape the business end to end.
2. They manage AI as a portfolio. Gartner's high performers track the return on each initiative and reallocate or stop weak ones; they reported positive returns on 81% of their AI initiatives, while low performers didn't know the return on 29% of theirs.
3. They make bigger, fewer bets. BCG's 2024 AI leaders scaled twice as many AI solutions as others and backed them with twice the digital investment.
4. They move quickly. In MIT NANDA's research, the fastest mid-market companies went from pilot to full implementation in about 90 days on average; large enterprises took nine months or longer.
5. They buy where it makes sense. MIT NANDA found external partnerships reached deployment about 67% of the time, against about 33% for internal builds, though the authors caution the figures are self-reported. The picture is shifting: in McKinsey's 2026 survey, 32% said their organisation had decided against buying software because agentic coding tools could build it in-house.
6. They govern early. Regular assessments, clear rules on who can use what, and human checks on high-stakes outputs make scaling safer, not slower. The broader framework is in AI governance, and the risk of staff using unapproved tools is covered in shadow AI.

What the data shows

A bank that scaled it: DBS

• Value: DBS says its data and AI initiatives delivered more than SGD 750 million of economic value in 2024, more than double the year before, across over 1,500 models and 370 use cases.
• 2025: about SGD 1 billion of economic value from more than 2,000 models across 430 use cases, according to its 2025 annual report.
• Caveat: economic value is DBS's own measure, not revenue, and the bank doesn't publish its method.

The reverse lessons are just as useful. Klarna said in February 2024 that its AI assistant was doing the equivalent work of 700 full-time customer service agents; by May 2025 it was recruiting human agents again, and its chief executive told Bloomberg that cost had been too dominant a factor and quality had suffered. In August 2025 Australia's Commonwealth Bank reversed 45 job cuts linked to an AI voice bot and apologised to the staff affected, after finding the roles weren't redundant. Gartner predicts that by 2027, half of organisations that planned to significantly cut customer service staff because of AI will abandon those plans.

Will AI agents end pilot purgatory or deepen it?

In the short term, probably deepen it. AI agents, which plan and carry out multi-step tasks across tools, need everything a chatbot pilot needs plus permissions, integration with more systems and monitoring of what they actually do.

• Cancellations. Gartner expects more than 40% of agentic AI projects to be cancelled by the end of 2027 because of escalating costs, unclear business value or inadequate risk controls. It also warns of agent washing, the rebranding of existing chatbots and automation tools as agents, and estimates that only about 130 of the thousands of vendors selling agentic AI are real.
• Scale is concentrated in big firms. In McKinsey's 2026 survey, 40% of respondents at organisations with more than $1 billion in revenue reported scaling AI agents, up from 27% a year earlier; at smaller organisations the figure stayed at 22%.
• Governance lags. Close to three-quarters of companies in Deloitte's survey plan to deploy agentic AI within two years, but only 21% have a mature governance model for agents.
• Running costs bite. About 20% of McKinsey's 2026 respondents said AI operating costs, including token costs, had constrained their AI use.

For how agents work and where they pay off, see agentic AI and AI agents for business.

What does AI pilot purgatory mean for the UAE?

By Microsoft's measure, the UAE uses AI more than any other country, but broad use isn't the same as scaled business value.

• Use is the highest in the world. Microsoft's latest AI diffusion update put the UAE first, with 73.3% of the working-age population using AI in June 2026, against 18.8% worldwide. That measures people using AI tools, not companies running AI at scale.
• Government has set a scaling target. In April 2026 the UAE Cabinet unveiled a framework to deploy agentic AI across 50% of government sectors, services and operations within two years, by 2028. In May it approved the federal framework and launched training for 80,000 government employees in agentic AI tools.
• Private-sector readiness lags. Korn Ferry's 2026 human-capital survey of 105 organisations across the six Gulf states found 49% piloting AI in selected functions, 20% deploying it across multiple business areas and only 1% describing themselves as fully AI-ready. Beyond integration, talent gaps (44%) and unclear ROI frameworks (37%) held back returns.
• Expectations are measured. KPMG's 2026 UAE survey, based on 70 local technology leaders, found 60% expect AI at scale within 12 months, against 68% globally.

The government programme builds in several things the research links to scaling: named accountability, redesigned processes, mass training and phased targets. For private companies in the UAE, the practical lesson is the same: widespread use is a starting point, not proof that AI is working.

What does it mean for real estate?

Real estate is a clear example of pilots outrunning results. JLL's 2025 global technology survey of more than 1,500 senior investors and occupiers across 16 markets found 88% of investors, owners and landlords and 92% of corporate occupiers piloting AI, but only 5% of occupiers had achieved all their AI programme goals. As recently as July 2023, fewer than 5% of companies had started AI pilots. In JLL's investor research, over 60% remained unprepared for scaled AI beyond pilots. BCG places real estate and construction at the lower end of its AI maturity curve.

In Dubai, the clearest public example of AI in operation comes from the regulator. The Dubai Land Department says its AI-enabled advertising governance platform had monitored more than 279,000 property listings on Property Finder, Dubizzle and Bayut by April 2025, and that 29% of the monitored listings were automatically modified. That's an operational result, not a return-on-investment figure, but it's the kind of measurable output private firms should aim for.

What this means

What this means for Dubai brokerages and developers. Pick one workflow with a clear number attached, such as response time to portal leads or viewings booked per enquiry, and run the pilot on real leads with a set end date. Connect it to your CRM from the start, and don't cut staff until volumes and quality are proven. Klarna and Commonwealth Bank both had to reverse course. For worked examples, see AI lead routing and the wider AI in real estate guide.

How do you get out of AI pilot purgatory?

Treat every pilot as the first phase of a production system, and be willing to stop the ones that won't get there. This health check helps spot the difference:

Question	Healthy pilot	Purgatory warning sign
Who owns it?	A business leader who owns the process and the budget	An innovation team with no business owner
What does success mean?	A baseline and a target on a business KPI, agreed up front	No agreed measure of success
What data does it use?	Production data, with access approved	A cleaned sample or demo dataset
How does it connect?	An agreed path into the CRM, ERP or core systems	A standalone tool, copy and paste
What happens next?	A date to scale, fix or stop	Open-ended extensions
Who has been trained?	The people whose work will change	Only the pilot team

Then work through the portfolio:

1. List every pilot with its owner, cost, KPI and decision date. Anything without an owner is a candidate to stop.
2. Choose fewer, bigger use cases tied to a process that matters to revenue or cost.
3. Build for production from day one: real data, the real integration path, and a security and risk review.
4. Redesign the workflow around the AI, including approvals and hand-offs.
5. Fix the data the use case needs, not all your data at once.
6. Budget for running costs, including tokens, monitoring and training, not just the build.
7. Decide on the date, and record why you scaled, fixed or stopped.

Final takeaway

AI pilot purgatory isn't a sign that AI doesn't work. It's a sign that most organisations treat AI as a series of experiments rather than a change to how work gets done. The companies that scale choose fewer use cases, redesign the workflow, fix the data, govern early and decide on a date. For UAE businesses, where everyday AI use is already among the highest in the world, the opportunity is to turn that use into measured, scaled results.