The 70% AI productivity myth: why most companies aren't seeing the gains

“Now consider the narrative you've been hearing from vendors, executives, and LinkedIn thought leaders: AI has collapsed software development costs by 70-90%. Development velocity is through the roof. If you're not seeing these gains, you're doing it wrong.

These two realities don't fit together. If even Karpathy feels behind, what hope does the average enterprise engineering team have?

The answer is uncomfortable: the 70-90% productivity claim is true for about 10% of the industry. For the other 90%, it's a marketing hallucination masquerading as data.”

A randomized controlled study by METR (Model Evaluation & Threat Research) found something that should terrify every CTO: experienced developers using AI tools took 19% longer to complete tasks than those working without them.

Not beginners. Not interns fumbling with ChatGPT. Experienced engineers. On codebases they knew. With tools designed to make them faster.

They got slower.

The Stack Overflow 2025 Developer Survey adds nuance. While 52% of developers report some positive productivity impact from AI tools, only a minority experience transformative gains. 46% now actively distrust AI output accuracy, up from 31% last year. The number-one frustration, cited by 66% of developers: AI solutions that are "almost right, but not quite", leading to time-consuming debugging.”

“This isn't learning a new library or framework. This is learning to work with something that is:

For experienced developers, this may actually be harder. They have decades of muscle memory around deterministic systems. They've internalized debugging strategies that don't apply when the "bug" is an LLM hallucination with no stack trace.

The data supports this. Only 48% of developers use AI agents or advanced tooling. A majority (52%) either don't use agents at all or stick to simpler AI tools. 38% have no plans to adopt them.”