More than Half of AI Projects Fail—but Not Because of Foundational Models



This is Part 1 of a three-part series on AI data readiness.

AI has become the defining technology investment of the last five years. Organizations are pouring resources into generative AI, predictive analytics, intelligent automation, and increasingly, agentic AI systems capable of making decisions and acting autonomously. But despite the enthusiasm (and the spending), many AI initiatives never deliver meaningful business value. 

The enthusiasm is real. So is the spending. But walk into most organizations and ask what actually made it to production, and the room goes quiet.

Here’s the uncomfortable truth I keep running into with enterprise leaders: The foundational model is never the problem!

The numbers back up the silence. A recent IDC study found that 54% of AI proof-of-concept projects never make it into production. More than half. Which means organizations experimenting with AI are spending the money and never seeing the return for half of it.

Why? The answer isn’t necessarily the model. It’s the data. More specifically, it’s the assumption that yesterday’s architectural decisions will still support tomorrow’s AI requirements. They won’t. I’ll say that plainly, because it’s the whole point of this series.

The AI infrastructure problem nobody talks about

When most organizations begin their AI journey, they focus on immediate needs:

  • Selecting models
  • Choosing a cloud service to deploy on
  • Testing initial use cases with sample data

This is a great way to get started. But it obscures a larger reality:

AI is evolving fast. But getting it into production requires more than simply writing an application.

Think about how fast the ground has moved. A few years ago, enterprise AI mostly meant predictive models built on structured data. Today, it’s generative applications chewing through massive volumes of unstructured content. Tomorrow, it’s agentic systems orchestrating workflows across your applications, your data sets, and your business processes all at once.

Each wave introduces new performance requirements, new governance challenges, and new data management demands. 

Translation: The architecture supporting today’s AI can become tomorrow’s bottleneck far sooner than anyone planned for.

Why AI readiness is a moving target

One of the most important findings in IDC’s research is that AI readiness is an ever-evolving maturity journey, as opposed to just a destination. 

IDC’s AI Readiness Index identifies four stages of maturity:

  1. Experimenting
  2. Practitioners
  3. Competent
  4. Masters

Here’s what separates the organizations getting real returns from the ones stuck in pilot. It isn’t that they’re running more AI projects. It’s that they focused on fueling their projects with “AI-ready data” and designed their “AI factories” to adapt—to changing workloads, to new technologies, to business requirements nobody has written down yet. Flexibility wasn’t an accident. They intentionally designed for it!

And look at the pace they’re designing against. New foundation models land monthly. Agentic architectures are rewriting how applications get built. Regulatory requirements shift by the week. Hardware generations turn over faster than any traditional enterprise refresh cycle. Data volumes keep exploding. In that kind of environment, infrastructure flexibility stops being a nice-to-have and becomes table stakes.

The real goal? Adaptability

Traditional infrastructure planning often focuses on optimization:

  • How do we maximize performance?
  • How do we reduce costs?
  • How do we simplify operations?

Those questions still matter. But AI adds one more, and it’s the one that decides who ships and who stalls: 

How fast can we adapt?

IDC put it about as bluntly as I would:

“No infrastructure investment made today can be assumed sufficient for AI workloads in 2028 or 2030. IT leaders must prioritize architectural flexibility, designing systems that can be extended, scaled, and reconfigured as AI technology matures.”

That’s a real shift in how to think about this. The question is no longer “does my infrastructure support the workload I have?” It’s “can my infrastructure evolve when the workload changes?” Because it will.

The 4 areas where flexibility matters most

1. Performance requirements

AI workloads don’t share a profile. Training a large language model needs sustained throughput to keep GPU clusters fed. Real-time inference lives and dies on ultra-low latency. Agentic systems need continuous access to dynamic, distributed data. Your infrastructure has to serve all of that at once—and rigid architectures tuned for a single workload tend to buckle the moment the next initiative shows up.

2. Data growth

The pool of enterprise data feeding AI keeps expanding: documents, images, video, telemetry, application logs, customer interactions, third-party sources. As those footprints grow, infrastructure has to scale without spinning up new silos, opening governance gaps, or piling on operational complexity.

3. Deployment models

Nobody is doing AI in one facility. Workloads increasingly span public cloud, private cloud, on premises, and the edge. Infrastructure has to move data across all of it—with performance, security, and governance intact the whole way.

4. Future AI workloads

Most organizations are still planning around today’s AI capabilities.

The challenge is that tomorrow’s most valuable use cases are yet to be discovered. Just as few organizations anticipated the rapid rise of generative AI five years ago, today’s infrastructure planners should assume future requirements will differ substantially from current assumptions.

The organizations that succeed will be the ones prepared for change.

What AI leaders are doing differently

IDC’s research shows that the most mature organizations architect infrastructure with future evolution in mind. Rather than treating modernization as a one-time project, they view AI readiness as an ongoing process involving people, processes, and technology maturing together.

These organizations tend to:

  • Design for extensibility instead of rigid platforms 
  • Eliminate siloed data repositories  
  • Prioritize data mobility and interoperability with a common data and control plane
  • Treat AI-ready data as the foundational element of their AI stack
  • Build security and governance into their data platform 
  • Continuously re-assess readiness against AI demands

In other words, they optimize for resilience, not model permanence.

Planning for change is planning for AI success

AI success isn’t decided by your models, your GPUs, or your budget. It’s decided by whether you can deliver AI-ready data and adapt your infrastructure as the technology, models, pipeline, workloads, and business priorities shift underneath you.

The organizations achieving the greatest returns from AI understand a simple reality: The future of AI remains fluid, and that’s exactly why flexibility matters.

But to adequately plan for change, you need to understand where your data readiness and infrastructure stands today—and how it may be limiting your AI investments. That’s what we’ll discuss in Part 2. 

Coming next in this series

Part 2: The Journey to AI-Ready Data
We’ll explore how to rethink your data estate in order to deliver refined, context-aware data as the critical raw material for your AI factory.