• Home
  • About me
  • Work
  • Blog
  • Contact
René Manikofski René Manikofski
René Manikofski

The AI Productivity Paradox

AI productivity, measured honestly

Developers felt 20 % faster with AI. They were 19 % slower.

That gap, from a 2025 study, is not a developer quirk. In McKinsey’s 2026 survey, 80 % of people say AI made them more productive, but only 37 % see any effect on company profit. That is the AI productivity paradox: the tools work, yet the gains rarely reach the business. This article looks at why, and how to measure the gains properly.

In this article

  • Between Hype and Collapse
  • The Perception Gap
  • Task Level vs. Company Level
  • The Real Cost
  • Why It Fails: No Owner
  • What This Means for UX Teams
  • How to Measure It
  • We Have Been Here Before
  • Bottom Line
  • Sources

Between Hype and Collapse

Two camps dominate the AI debate. One warns of mass job loss. In May 2025, Anthropic CEO Dario Amodei said AI could wipe out half of all entry-level white-collar jobs and push unemployment to 10–20 % within five years. The other camp calls it all hype.

Neither holds up well. A year later, in May 2026, OpenAI CEO Sam Altman said he had been “pretty wrong” about the social and economic impact, and Amodei softened his warning, both ahead of expected IPOs. Economists such as Daron Acemoglu at MIT expect modest gains: around half a percent more productivity over ten years.

The evidence points to the middle. AI changes single tasks today. It has not caused mass job loss so far. And companies change slowly, mostly where they prepared first.

Who makes an AI forecast, and for whom, shapes the number. Read it with the same care as any sales pitch.

The Perception Gap

Vendor studies from GitHub, Microsoft and Google report roughly 20 to 55 % faster work. But they come from the companies selling the tools, often use selected tasks, and participants know they are being measured.

In July 2025, the independent research group METR ran a randomised controlled trial: 16 experienced open-source developers, 246 real tasks from their own projects. With AI tools they took 19 % longer. Yet they believed they had been 20 % faster.

METR study, July 2025

−19 %

measured change in speed with AI tools

+20 %

how much faster the same developers felt

39 pts

gap between what was measured and what was felt

An important update: in February 2026, METR said this result is now outdated. With newer tools, speed-ups look likely, but its new data is too uncertain to say how large they are. One reason: more and more developers refused to take part without AI, which skews the sample.

So the slowdown itself may be history. The lesson is not. People judge their own AI productivity badly, and they misjudge it in the tools’ favour. Any company that measures AI success by asking “do you feel more productive?” is measuring the wrong thing.

The question is not whether AI can make people faster. It is whether your company can tell the difference between feeling faster and being faster.

Task Level vs. Company Level

Most companies mix up two questions.

At task level, AI helps. Drafts, summaries, reformatting data, writing test cases: people report 20–30 % gains on routine work. But these are self-reports, the same kind of judgement that proved unreliable above.

At company level, the picture is different. McKinsey’s 2026 survey puts the paradox in four numbers:

The AI productivity paradox in four numbers

use AI in at least one business function88 %
say AI made them personally more productive80 %
see any effect of AI on company profit37 %
are high performers with a real share of profit from AI6 %

Source: McKinsey, The State of AI 2026. Different questions from the same survey, so read it as a gap, not a strict funnel.

80 % feel more productive. 37 % see it in profit. That gap is the AI productivity paradox.

The pattern repeats elsewhere: 42 % of companies abandoned most of their AI initiatives in 2025, up from 17 % the year before (S&P Global). The tools work. The gap is in the organisation.

Software teams show where the problem goes. Faros AI analysed data from more than 10,000 developers. Teams with high AI use merged 98 % more pull requests, but review time rose 91 % and bugs per developer rose 9 %. Delivery at company level did not improve. Faros’ 2026 follow-up with about 22,000 developers points the same way, only more sharply: 54 % more bugs per developer, review times around five times longer, and nearly a third of pull requests merged without any review.

AI does not remove the bottleneck. It moves it. If you don’t measure where it went, you optimise the wrong thing.

The Real Cost

AI budgets usually start with licences, typically €20–60 per user and month. They are visible, but only one part of the cost.

What AI really costs

1 — Licences and usage fees
Licences are fixed. Usage-based fees grow fast with automated, multi-step workflows (agents) and often surprise on the first invoice. More on the token side below.
2 — Training
Without integration into real workflows, many people use AI only now and then. That rarely pays back. Training needs repeating as the tools change.
3 — Governance and quality checks
Who checks AI output, and where? Which tasks move between roles? Open questions turn into hidden costs: rework, errors in production, lost trust.
4 — Review effort
More output means more to review, as the developer data shows. Plan for it.
5 — Measured results
The return needs the same rigour as the cost. Activated licences are not a return. A measured change against a baseline is.

The second paradox. Per token, AI keeps getting cheaper. Yet bills keep rising, because agents use far more tokens than a simple chat. EY’s example: a customer-service interaction that cost about 4 cents in 2023 can cost around $1.20 with a 2026 agent setup, roughly 30 times more.

Some companies now throttle access: monthly spending limits per employee, caps on tokens, or a switch to cheaper models. That saves money, but it can remove the very gains they paid for. Restrict staff to weaker models, and the conclusion follows that “AI doesn’t work”. Give everyone unlimited access, and the invoice arrives.

Both mistakes have the same root: steering AI by budget lines instead of by results per task.

The better lever is to measure cost per completed task: cheaper models for routine work, top models where the quality pays back. How token bills grow, and why they surprised so many companies, is covered in detail in Tokenmaxxing: The Bill Nobody Showed You.


Why It Fails: No Owner

One pattern explains most failures: AI is treated as an IT purchase, not as a change in how work gets done, with someone accountable for it. Google’s DORA research with nearly 5,000 tech professionals calls AI an amplifier: it magnifies the strengths of well-run organisations and the problems of struggling ones.

Often there is no defined use case, no quality check, no measurement. Tools multiply, and nobody knows what they produce. The same maturity gap exists in UX teams: UX Maturity.

Licences are bought centrally. Use is left to individuals. That gap is where the return disappears.

PwC’s 2026 AI Performance Study found that 20 % of companies capture 74 % of AI’s economic value. What sets them apart is not which tools they bought, but how they use them. They work like this:

How the leaders work

Few use cases, deep integration
Two or three use cases with clear owners, quality checks and measurement from day one. Depth beats breadth.
Governance from the start
Someone owns the answer to: what does good output look like here, and who checks it?
Outcomes, not activations
Success is a measured change in a business number, not the share of staff with an account.
Involve employees early
The people doing the work know where time goes and where AI could help. Ask them before the rollout.

What This Means for UX Teams

For UX work, hard data is thin. Most evidence is self-reported: 20–30 % faster synthesis, analysis and production.

AI helps most with the parts of UX that produce artefacts: transcripts, notes, test tasks, report layouts. The parts that need judgement stay with people: what to research, what findings mean, which design decision is right. More on this in AI & UX: The Missing Layer.

Nielsen Norman Group’s State of UX 2026 makes a similar point: work limited to assembling screens is the most exposed, while taste, context and judgement gain value.

The risk is not replacement. It is lower average quality: more screens, more reports, more personas, less signal. Use AI to run more research, not less.

AI is also only as good as the data it can use. Gartner found that 63 % of organisations lack, or are unsure about, AI-ready data practices, and expects 60 % of AI projects without such data to be abandoned by the end of 2026. Every company can rent the same model. Its own clean, connected customer data is what others cannot copy, the same argument as for research: Synthetic Personas & Data.

The bottleneck has moved from producing screens to deciding what is right. That is a UX argument, and the data now supports it.

How to Measure It

Companies that get real returns are not smarter about AI. They are smarter about measurement, and they do the groundwork before the rollout. That is still rare: a September 2026 review of 61 studies found none that measured enterprise AI returns with audited costs and benefits.

You cannot measure improvement without a baseline. If you don’t know how long a task took before AI, you cannot know whether it is faster now.

Five steps to a productivity proof

1 — Set a baseline before rollout
Pick two or three typical tasks. Measure time, errors and revision rounds for at least four weeks before AI tools arrive.
2 — Measure outcomes, not hours saved
Saved hours fill up with other work. Track cycle time, error rate, response time or revenue per case instead, and set them against the cost per completed task, including tokens.
3 — Measure the whole workflow
A faster writing step means nothing if review time doubles. Follow the bottleneck.
4 — Start with one or two use cases tied to a business number
Not "everyone uses the tool", but "support tickets are solved in two hours instead of four".
5 — Build quality checks into the rollout
Decide who reviews AI output and what good looks like before the first team goes live.

One use case, properly baselined, with a business number and a quality check, proves more than a company-wide rollout measured by survey. It is also what makes the investment defensible at board level. For the UX side of that argument: UX & Revenue.

The goal is not a productivity feeling. It is a productivity proof.

We Have Been Here Before

In 1987, economist Robert Solow wrote: “You can see the computer age everywhere but in the productivity statistics.” Historian Paul David found the same pattern with electricity: factories only became more productive decades later, once they reorganised around the new technology.

AI will not take decades. There are no factories to rebuild; it arrives as software. But the slow part was never the technology. It is redesigning how work gets done, and that is still slow and political. Buying tools is a budget decision. Changing how people work is a power question.

Easy to use is not the same as easy to get value from.

Bottom Line

Treat AI as a change project, not a purchase. Pick a few use cases, give each an owner, measure against a baseline and look at the whole workflow.

This matters even more with AI agents, the current wave: tools that take over whole multi-step tasks. In McKinsey’s 2026 survey, 40 % of large companies are scaling them, up from 27 % a year earlier. Agents multiply output, review effort and token bills alike. The same rules apply, only faster.

Companies that follow these rules will have proof within a year. The others will have licences and a good feeling.

Quick check

Think you can tell feeling faster from being faster?


Sources

  • METR — Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (2025)
  • METR — Full paper, arXiv:2507.09089 (2025)
  • METR — We are Changing our Developer Productivity Experiment Design (2026)
  • Axios — AI jobs danger: Sleepwalking into a white-collar bloodbath (2025)
  • TechSpot — Sam Altman is “delighted” his AI jobs prediction was wrong (2026)
  • Acemoglu — The Simple Macroeconomics of AI, NBER (2024)
  • GitHub — Quantifying Copilot’s impact on developer productivity (2022)
  • McKinsey — The State of AI: Global Survey (2026)
  • CIO Dive — AI project failure rates are on the rise, S&P Global data (2025)
  • Faros AI — The AI Productivity Paradox Research Report (2025)
  • EY — Agentic AI enterprise token costs (2026)
  • 404 Media — Companies are throttling employees’ AI use because it’s too expensive (2026)
  • AI Weekly — AI got too expensive, so companies are moving to cheaper models (2026)
  • Faros AI — AI Engineering Report 2026: The Acceleration Whiplash
  • DORA — State of AI-assisted Software Development (2025)
  • Enterprise AI at the Decision Point — systematic review, preprint (2026)
  • PwC AI Performance Study, via Resultsense (2026)
  • Gartner — Lack of AI-Ready Data Puts AI Projects at Risk (2025)
  • Nielsen Norman Group — State of UX in 2026
  • Paul A. David — The Dynamo and the Computer, American Economic Review (1990)

Robert Solow’s quote is from his book review “We’d Better Watch Out”, New York Times Book Review, 12 July 1987.

via GIPHY

René Manikofski is a UX leader with 15 years of experience in e-commerce and digital product design across Europe. All articles are based on personal professional experience and supported by AI in writing.

8
Like this post
  • Previous PostThe Cancellation Moment

© 2026 René Manikofski – Made with love in Berlin – Germany – Impressum

Copy