Summary
Companies are spending huge amounts on AI tokens, often cutting jobs to pay for them. But many are finding that cutting people does not lead to better returns. The smarter approach is to reduce token costs through engineering, not layoffs. By using techniques like prompt caching and choosing the right AI model for each task, businesses can save money while keeping their teams intact.
Main Impact
The core problem is that many companies treat their token budget as fixed and their workforce as flexible. They cut jobs to free up money for AI, but this often fails to improve results. A Gartner survey found that 80% of companies that cut headcount for AI saw no link to better returns. The real solution is to make the token budget flexible through smart engineering, not by losing skilled workers.
Key Details
What Happened
Nvidia CEO Jensen Huang said on the All-In Podcast that if a $500,000 engineer uses less than $250,000 in AI tokens per year, he would be "deeply alarmed." Nvidia aims for a $2 billion yearly token bill for its engineers. This shows how companies are shifting money from salaries to AI tokens.
Important Numbers and Facts
The four largest cloud companies plan to spend about $700 billion on AI in 2026, nearly double last year. AI is now the top reason for U.S. job cuts for four months in a row. Uber gave 5,000 engineers AI coding tools in December and used up its entire 2026 AI budget by April. The company then put a $1,500 monthly cap on each engineer's token use.
Background and Context
Many businesses see AI as a way to cut costs by replacing workers. But this approach often backfires. Klarna replaced 700 customer service roles with an AI assistant, but customer satisfaction dropped. The company later rehired humans for tasks needing judgment. Gartner predicts that by 2027, half the companies that cut customer service staff for AI will rehire them.
Public or Industry Reaction
Gartner analyst Helen Poitevin said, "Workforce reductions may create budget room, but they do not create return." Uber's COO Andrew Macdonald admitted that while 70% of code is now AI-generated, the link to customer benefits is missing. Klarna's CEO Sebastian Siemiatkowski said the AI-only model led to "lower quality, and that's not sustainable."
What This Means Going Forward
Companies can cut token costs without cutting people. Simple steps include prompt caching, which can reduce costs by up to 90% for repeated inputs. Using smaller, cheaper models for routine tasks and batch processing for non-urgent work also helps. Open-weight models offer further savings. These methods are like turning off lights in empty rooms—they save money without hurting the team.
Another concern is the loss of junior developers. Stanford found that employment for software developers aged 22 to 25 fell nearly 20% from 2024 levels. This removes the training ground for future senior engineers. Companies that save on tokens can use that money to keep hiring and training new talent.
Final Take
The companies that will succeed are not those that spend the most on tokens or cut the most people. They are the ones that treat the token budget as flexible, use engineering to reduce costs, and invest the savings in the people who make AI valuable. As Huang's comment suggests, the goal is not to replace engineers but to make them more productive with AI.
Frequently Asked Questions
What is prompt caching and how does it save money?
Prompt caching is a technique where repeated parts of a request, like system instructions, are processed once and reused. This can cut costs by up to 90% because the AI does not have to reprocess the same text each time.
Why do companies cut jobs for AI if it does not improve returns?
Many companies see AI as a way to reduce costs quickly. But cutting jobs often removes valuable knowledge and skills. The savings from layoffs may not lead to better results if the AI is not used effectively.
How can a company reduce its AI token budget without layoffs?
Companies can use several methods: prompt caching, choosing smaller models for simple tasks, batch processing for non-urgent work, and using open-weight models. These steps can cut token costs by 50% or more without losing staff.
