Microsoft is drawing a line from the semiconductor industry's oldest obsession, yield, straight into the heart of the AI buildout. In a new post, Azure hardware chief Rani Borkar argues that with agentic AI tasks burning through 3,400 times more tokens than a simple chat query, the industry can no longer just throw more silicon, power and memory at the problem. It has to get smarter about what actually comes out the other end.
The semiconductor industry has spent 60-plus years obsessing over one word: yield. Not elegance, not roadmaps, just one question, according to Rani Borkar, who leads Microsoft's core hardware and infrastructure organization for Azure. 'What useful output did we produce?' That question, she argues in a new post on the Official Microsoft Blog, is exactly what the AI industry needs to start asking itself, and fast.
The timing isn't subtle. Hyperscalers have poured capital 'on a scale once reserved for nations' into gigawatt-scale data centers and record-breaking fabs, Borkar writes, and the bill is starting to look unsustainable if the only answer to every new bottleneck is simply more. More silicon in the package. More memory beside it. More power to feed it. 'When each new gain requires more input than the one before it, we are on a treadmill,' she writes. 'It moves only as long as we keep adding to it.'
[Image: Line chart showing AI adoption curve compared to electricity, radio, internet and cellphones, sourced from Microsoft's AI Diffusion Report]
What makes the math especially uncomfortable is where AI usage is actually headed. Chat-based prompts, the thing most people picture when they think of tools like ChatGPT, are giving way to agentic workflows where a system reasons, plans, calls tools and executes multi-step tasks over minutes or hours. According to Microsoft's own AI Diffusion Report cited in the post, a single agentic task can consume more than 3,400 times the tokens of a typical chat exchange, even though global AI penetration is still only around 18% of the working population. In other words, the hardest part of the scaling curve hasn't even started yet, and the infrastructure is already straining under power caps, denser racks and tightening memory budgets.
Borkar's answer isn't a single breakthrough chip or cooling trick, it's a change in how Microsoft engineers the whole stack. She points to two lessons the company has learned building and running AI infrastructure at scale: constraints are rarely solved in the layer where they show up, and once you attack a bottleneck across the entire stack, tradeoffs that looked baked-in often turn out to be architectural artifacts you can just design away.
The clearest example is memory. Borkar frames it not as a supply chain problem but a systems problem, since inference workloads now need to hold bigger models, preserve longer context windows and feed compute fast enough to keep pace with agentic loops that can run for hours. Building the Azure Maia inference platform, Microsoft's team found that no single fix, not compression, not smarter memory hierarchies, not silicon tuned for data movement, solved the bottleneck alone. Stacked together, though, they squeezed more usable intelligence out of the same memory footprint. 'That is useful yield,' Borkar writes, 'not simply adding bytes but getting more useful intelligence from every byte we already have.'
Networking gets the same treatment. Rather than bolting together separate scale-up and scale-out fabrics, Microsoft built Maia around a two-tier scale-up network with NIC functionality baked directly into the chip and a custom transport layer, a design choice Borkar says simplified programming and cut the amount of network hardware needed, lowering costs for the whole system in the process.
Then there's power, arguably the tightest leash on the entire industry right now. Racks have jumped from tens of kilowatts to hundreds, and campuses now run at gigawatt scale. Microsoft's Azure Cobalt 200 Arm-based server chip gives every core its own voltage and frequency controls paired with per-VM power capping in software, letting Microsoft cram more servers into the same power envelope without sacrificing performance on critical workloads.
[Video: YouTube embed titled 'The next curve: Turning AI infrastructure into useful intelligence']
None of this is charity. Borkar is explicit that the endgame is economic: efficiency is what makes it possible to deploy AI broadly enough, and cheaply enough, that a scientist, a clinician, a student or a small business owner can actually put it to use. She calls that 'full yield,' the point where the intelligence generated by all this infrastructure translates into value for the people paying for it, not just impressive benchmark numbers. It's a framing that puts Microsoft's hardware bets, and its ongoing rivalry with Google, Amazon and Nvidia over custom silicon, squarely in economic terms rather than raw compute terms, which is probably exactly where investors want the conversation to go as AI capital expenditure keeps climbing into 2027.
Whether or not 'yield' becomes the industry's new buzzword, Borkar's post is a signal of where the AI infrastructure conversation is heading next. The era of simply announcing bigger data centers and bigger power deals is running into physical and financial limits, and companies that can prove they're squeezing more usable intelligence out of every chip, watt and dollar are going to have a much easier time justifying the next round of spending to investors. For Microsoft, that means leaning harder into custom silicon like Maia and Cobalt as proof points. For everyone else building AI at scale, it's a preview of the metric they'll increasingly be judged on.
