The MIT 95% AI Pilot Study Rests on 52 Interviews. Here Is What That Supports

The claim that 95% of AI pilots fail comes from a preliminary MIT NANDA report built on 52 interviews, 153 conference surveys and a review of public announcements. Read the MIT 95% AI pilots fail study on its own terms and the figure describes organisations, not pilots. Inside its own funnel, 25% of organisations that piloted a custom tool got it into production.

This post is written for the ops lead or head of delivery who runs AI pilots and keeps hearing that 95% of them fail. That reader needs to know which part of the figure applies to their own pilot plan.

The short answer. The MIT NANDA study supports a narrow claim: in a small, non-random sample, few organisations had moved a custom generative AI tool from pilot to production with a measurable effect within six months. It does not support a market-wide failure rate, and it does not measure the failure rate of individual pilots at all.

What the MIT 95% AI pilot study actually claims

The report is titled The GenAI Divide: State of AI in Business 2025. It was published in July 2025 by Project NANDA, a research initiative at the MIT Media Lab. The authors are Aditya Challapally, Chris Pease, Ramesh Raskar and Pradyumna Chari, and the cover describes the work as preliminary findings.

The executive summary pairs two numbers. Enterprises had put an estimated $30 billion to $40 billion into generative AI, and 95% of organisations were getting zero return. That pairing is what turned a 26-page working paper into a market event in August 2025.

The second number travelled further than the first. Fortune's coverage ran under a headline about 95% of generative AI pilots at companies failing. That framing, pilots rather than organisations, is the version most people now quote, and it is not quite what the report measured.

One camp treated the report as proof of a bubble and the other dismissed it as junk. Both skipped the method section, where the useful information sits.

The report's own core argument is not really about failure. It is about a learning gap: most generative AI systems in its sample did not retain feedback, adapt to context or improve over time. The failure statistic is the hook. The learning gap is the thesis, and it is the more defensible of the two.

How the MIT NANDA sample was built

A failure rate is only as good as the population it was measured on. The MIT NANDA study used three sources of evidence, and each one has a different selection problem. None of the three is a random sample of companies.

52 structured interviews

The interviews were held with representatives of 52 organisations across industries. Company names and quotes were anonymised to comply with disclosure policies. That is normal practice, but it means nobody outside the team can check which firms took part.

The size matters for arithmetic, not just credibility. In a group of 52, one organisation is worth 1.9 percentage points. A 5% success share in that group is 2.6 organisations. A single company changing its answer moves the figure by about two points.

Early coverage made this worse. Fortune described the research as 150 interviews, a survey of 350 employees and 300 public deployments. The report states 52 and 153. The inflated version was repeated widely, so many readers believe the sample is roughly three times larger than it is.

153 survey responses from four conferences

The survey reached 153 senior leaders at four major industry conferences. Conference attendees are a self-selected group. Their employers pay for them to attend technology events, which tilts the sample toward firms already spending on it.

Selection of that kind can push the answer in either direction. Attendees may be further along than the average company, which would flatter the success rate. They may also be the people sent to find answers because a programme is stuck, which would depress it. The report gives no way to tell which effect dominates.

300 public initiatives

The third source is a systematic review of more than 300 publicly disclosed AI initiatives. Public disclosures are press releases, earnings call mentions and vendor case studies. Companies announce launches. They rarely announce quiet shutdowns, so a review of announcements sees the start of a project far more often than its outcome.

The research ran from January to June 2025. Many enterprise generative AI programmes only moved past experiments in 2024, so a mid-2025 snapshot catches a large share of them before any six-month review could have completed.

How the MIT NANDA study defined a pilot and a failure

Definitions decide headline numbers, and this report uses a strict one. That is a legitimate choice, but it is a choice, and it should travel with the number.

Success meant production, KPIs and a six-month clock

Ray Poynter read the PDF closely. By his reading, the report defined success as deployment beyond pilot, with measurable KPIs, and ROI measured six months after the pilot. For task-specific tools, a successful implementation was one that users or executives described as causing a marked and sustained productivity or P&L impact.

Two things follow. First, a pilot that worked technically but was never rolled out counts as failure. Second, the evidence of impact is what an interviewee or survey respondent said, not an audited figure from a finance system. Poynter reports that the authors themselves called the figures directionally accurate and based on interviews rather than official company reporting.

The six-month clock and the nine-month rollout

The detail I think matters most comes from inside the report. Coverage of the findings says mid-market organisations moved from pilot to full implementation in about 90 days, against nine months or longer for large enterprises.

Put that next to a test that looks for return six months after the pilot. A typical large enterprise in this sample is still implementing when the clock runs out. The study's own timing data predicts that large companies will score badly on its own test, whatever their eventual return.

The authors appear to know this. Their limitations note warns that a six-month window may be too short to judge complex enterprise deployments. That warning belongs next to the 95% figure every time it is quoted.

The denominator problem in the MIT 95% figure

A percentage needs a base, and this report uses several bases for what reads as one result. The 5% describes organisations in one place, pilots in another and custom tools in a third.

The funnel for task-specific enterprise tools is quoted in coverage as follows: 60% of organisations evaluated these systems, 20% reached the pilot stage and 5% reached production. For general-purpose tools such as ChatGPT and Copilot, over 80% had explored or piloted them and nearly 40% reported deployment.

The 25% hiding in the funnel

If the stages are nested, which is how a funnel normally works, the arithmetic is simple. Of the 20% of organisations that piloted a custom tool, 5% reached production. That is 25% of piloting organisations, or a pilot-stage failure rate of 75%. Of organisations that evaluated such tools, 8% got one into production.

That is still a poor result. A three-in-four failure rate at the pilot stage is a serious finding. But it is a different finding from 95%, and it is the one an ops lead should compare against.

The general-purpose numbers point the other way. Half the organisations that explored tools such as ChatGPT or Copilot deployed them. Poynter notes the report also gives a pilot-to-implementation rate of around 83% for generic chatbots. The study's failure story applies to custom enterprise builds, not to generative AI in general.

The report itself blurs this. Coverage quotes it saying that just 5% of integrated AI pilots are extracting millions in value. That sentence uses pilots as the base, while the funnel uses organisations. That is how a statistic about organisations became one about pilots.

What 52 interviews can and cannot support

Small qualitative samples are useful. They are how researchers find mechanisms, the reasons something happens, before anyone can count it properly. The mistake is asking a qualitative sample to do a quantitative job. Here is how I would sort the report's findings.

Even a random sample of 153 would carry real uncertainty. At a 5% result, the standard 95% confidence margin for 153 responses is about plus or minus 3.5 points, so anywhere from roughly 1.5% to 8.5%. At a 50% result the margin is about 7.9 points. That arithmetic assumes random selection, which a conference sample does not have, so the true uncertainty is wider.

The build-versus-buy finding shows how fast numbers drift. Fortune reported that vendor partnerships succeed about 67% of the time, while internal builds succeed only one-third as often. One-third as often is about 22%. Other summaries report about 33%. Both trace to the same paragraph, and both get quoted as fact.

For a structured way to run this kind of check on any vendor claim, the method in the guide to reading AI ROI case studies forensically applies directly. A single report with an unclear base fails the same tests a weak case study does.

How the MIT figure compares with other AI pilot surveys

The MIT number is the most extreme of the widely cited pilot statistics, but it is not an outlier in direction. Every major source finds that a large share of AI experiments never reach production. They disagree on how large, and the disagreement mostly tracks what each one counted.

The two sources that count proofs of concept directly are the closest comparison to a pilot failure rate. IDC's research with Lenovo found that for every 33 AI proofs of concept launched, only four graduated to production. S&P Global found the average organisation scrapped 46% of its AI proofs of concept before production.

Those two land either side of the 75% implied by MIT's own funnel. Taken together, I think the honest summary is that somewhere between half and nine in ten enterprise AI experiments do not reach production. Anyone giving you a single precise figure is overstating what is known.

The Wharton figure is the outlier on the optimistic side, and for a structural reason. It asks leaders whether their generative AI investment is returning value overall, not whether a specific experiment succeeded. Wharton also reports that 72% of its respondents formally measure generative AI ROI, which is a different and arguably more useful signal.

The same pattern, a gap produced by different questions rather than different realities, drives the disagreement between vendor and consultancy surveys. It is set out in the comparison of NVIDIA's and McKinsey's AI ROI surveys, and the same three questions about sample, metric and threshold work here.

Where this reading of the MIT NANDA study is weakest

Here is where my argument is softest, judged by the standard I applied to the report.

I could not extract the report text directly

Every attempt to read the PDF in this session returned unreadable binary data. The method details here come from coverage that quotes the report, from Ray Poynter's close reading of it and from a mirror quoted in search results. Those sources agree on 52 interviews, 153 responses and the funnel stages. The exact wording of the limitations note is still second-hand.

My 25% figure also rests on an assumption. It treats the 60%, 20% and 5% stages as nested shares of the same organisations. If the report computed each stage on a different subset, the ratio would change. Coverage presents the stages as a funnel, and I have read them that way.

The direction of the finding is probably right

The strongest counter to my argument is that it does not change the practical conclusion much. Whether the pilot-stage failure rate is 75%, 88% or 95%, most custom enterprise AI experiments in 2025 did not reach production with a measured effect. IDC and S&P Global both support that, with larger samples than MIT's.

There is also a fair point about incentives running both ways. Poynter notes the report came from a project with a commercial interest in the agent systems it recommends. Many of its loudest critics sell AI services and benefit from a lower failure number. Neither interest proves anything, and I would hold both in mind.

Writing an AI pilot success definition that survives scrutiny

The most practical lesson from the MIT NANDA study is not about the market. It is that a success definition decides the result before the pilot starts. If your team has not written one down, your pilot will be judged against whichever definition the loudest person in the room prefers.

I would borrow the report's strictness and fix its timing. Production deployment with measured KPIs is a good bar. A fixed six-month clock for every pilot is not, because a 90-day mid-market rollout and a nine-month enterprise rollout cannot sensibly face the same deadline.

A workable definition has four parts. Name the unit, such as one workflow or one team. Name the metric, such as cost per ticket or cycle time, with a baseline measured before the pilot. Set a review date tied to the expected rollout length. Write down the result that would end the project.

That last part is the one teams skip. Kill criteria agreed in advance stop a pilot from drifting into the permanent limbo the report describes. A fuller treatment is in the guide to setting kill criteria for AI projects, and the common ways pilots stall are catalogued in the analysis of agent pilot failure modes.

Finally, measure the base you will report. If you run five pilots and two reach production, say 40% of pilots. Do not say 40% of the organisation. The MIT study is a reminder that the base you choose becomes the story others tell.

Frequently asked questions

Where to start with your own pilot numbers

Take the AI pilots your team has run in the past 12 months and count them two ways. First, divide pilots that reached production by pilots started. Then divide the same count by teams that could have used the tool. If the two numbers differ a lot, you now know which one belongs in a board slide.

Then pick the pilot that is currently furthest from a decision. Write its success definition in four lines, with unit, metric, review date and kill condition, and send it to the sponsor before the next status meeting.

Related reading