The ROI of AI for software engineers is not a PR count

aiengineering managementmetricsbudget

The Pragmatic Engineer asked what's the ROI of software engineers' spending on AI, with data from 27 companies plus CodeRabbit and Gitar. Spend goes from under $300 to over $3,000 per dev per month, PR counts are roughly doubling, and the honest answer on the ROI of AI for software engineers is that few companies can give one yet.

So here is what I would track:

  • Time-to-market: days from "we decided to build this" to "customers use it", on real features.
  • Backlog in vs out: are we finally closing more than we receive?
  • Headcount plan: did we open fewer roles than planned, or take on a project we would have refused? If neither, the speed went nowhere useful.
  • Cost of quality: incidents, rollbacks, review hours. Slop is cheap to write and expensive later.

What I would refuse to put in front of a CEO: PRs per developer, lines of code, and tokens spent as a sign of "adoption". All three go up when things get worse. Show a CEO that PRs doubled and they will ask why the roadmap didn't. Report token spend as success and you are celebrating your own invoice.

I'm not saying the gains are fake. The two-person .NET shop clearing a ten-year backlog mentioned in the article sounds very real to me. But engineering management has to translate it into delivery and cost before 2027 budgets get signed, because "we make more PRs now" is not an answer anyone outside engineering should accept.