TECH

Microsoft Maia 300 AI chip arrives with monstrous 288 GB HBM4 memory and 18 TB/s bandwidth

Microsoft has unveiled the Maia 300, its third-generation AI accelerator featuring 288 GB of HBM4 memory, 18 TB/s bandwidth and up to 13.8 PFLOPS of FP4 compute performance. The chip uses an advanced multi-chiplet design built around TSMC's N2P manufacturing process and will enter production deployment in 2027.

Microsoft has officially unveiled the Maia 300, its latest custom-designed AI accelerator intended to reduce the company's dependence on Nvidia and AMD hardware. It arrives shortly after OpenAI debuted its in-house silicon, Jalapeno.  The chip succeeds the Maia 200, which launched earlier this year. The first Maia chip, the Maia 100, surfaced in 2023. 

The Maia 300 uses a multi-chiplet design comprising a main compute section manufactured using TSMC's advanced N2P (2 nm) process. The package also incorporates separate memory and I/O chiplets, allowing Microsoft to manufacture different components using processes suited to their respective requirements. According to Reuters, some of the supporting silicon uses a less expensive 3 nm manufacturing process.

The accelerator delivers up to 13.8 PFLOPS of FP4 compute performance and 6.9 PFLOPS at FP8 precision. Its SoC power rating stands at 950 W, substantially higher than the Maia 200's 750 W rating. Microsoft has equipped the Maia 300 with eight HBM4 stacks, offering a combined 288 GB of memory and 18 TB/s of bandwidth. Each stack contributes 36 GB of capacity, with the HBM4 memory operating at 9 Gb/s per pin. For comparison, the Maia 200 features 216 GB of HBM3E memory with 7 TB/s of bandwidth, making the new accelerator useful for memory-intensive inference workloads.

Microsoft has also upgraded the accelerator's interconnect capabilities. The Maia 300 delivers 3.2 TB/s of bidirectional scale-up bandwidth and uses a two-tier Ethernet-based network with a custom AI transport protocol. A single rack accommodates 96 Maia 300 accelerators, while the scale-up architecture supports clusters containing approximately 12,000 accelerators.

Beyond its raw specifications, Microsoft claims Maia 300 can deliver 40% more tokens per megawatt than leading GPU-based AI systems. A fully populated 96-accelerator configuration reportedly offers up to seven times the rack-scale token throughput of Maia 200. However, the company has not provided sufficient details about the competing GPU configurations to independently establish the extent of its efficiency advantage.

Microsoft intends to begin deploying Maia 300 in production environments in 2027, with plans to eventually scale the infrastructure to multiple gigawatts of capacity. Reuters reports that the company could deploy more than one million Maia 300 chips next year, highlighting its ambitions to become less dependent on third-party AI accelerators.

Chiplet design and efficiency breakthroughs...Architecturally, the Maia 300 takes a fundamentally different technical approach from Nvidia. Nvidia's AI accelerators have traditionally centered on a single large compute die surrounded by high-bandwidth memory. The Maia 300, by contrast, employs a chiplet design, splitting compute, memory controllers, and input/output functions into multiple dies integrated into a single package.

The core compute chiplets are fabricated on Taiwan Semiconductor Manufacturing's (2330.TW) most advanced 2-nanometer process (N2P node), while other chiplets handling data movement and transmission use the more cost-effective 3-nanometer process. This hybrid process strategy allows Microsoft to pursue maximum performance on critical compute units while controlling costs on auxiliary functions.

On the memory front, the Maia 300 packs eight stacks of HBM4 high-bandwidth memory, totaling 288GB of capacity with transfer bandwidth of 18TB per second. By comparison, the previous-generation Maia 200 featured 216GB of HBM3E memory with 7TB per second of bandwidth. The new chip delivers roughly 33% more memory capacity and 157% more bandwidth — a critical upgrade for modern AI workloads that require long-context processing and multi-step reasoning.

In terms of raw compute, the Maia 300 delivers up to 13.8 PFLOPS at FP4 precision and 6.9 PFLOPS at FP8. Per-chip power consumption is approximately 950 watts, up from the Maia 200's 750 watts, but Microsoft emphasizes that the real benefit comes from system-level optimization.

40% higher output per megawatt...Scott Guthrie, executive vice president of Microsoft's Cloud and AI group, said the Maia 300 processes approximately 40% more tokens per megawatt of power consumed than "leading GPU systems." In a fully loaded rack configuration with 96 accelerators, rack-level token throughput reaches seven times that of the Maia 200.

However, Microsoft did not specify which GPU system served as the comparison baseline. Nvidia's latest GB300 AI server rack packs 72 GPUs with power consumption of roughly 142 kilowatts, while the Maia 300's 96-accelerator rack draws approximately 167 kilowatts. With specification details of the comparison target not fully disclosed, outside observers have difficulty independently verifying the actual substance of the claimed 40% efficiency advantage.

Rani Borkar, corporate vice president of Azure Hardware Systems and Infrastructure, told reporters: "We believe we have built the best product for these workloads. We will back that confidence with action by deploying Maia at gigawatt scale starting next year." She added that through holistic system optimization, Microsoft aims to ensure every watt of power, every dollar invested, and every byte of memory supports more AI compute.

Prioritizing internal services, not external rental...Unlike Google and Amazon, which have opened their in-house chip compute capacity to cloud customers, Microsoft has no current plans to rent out Maia compute directly to external clients. Borkar was blunt: "Our own AI services business is large enough. We want to serve ourselves first."

The logic of this approach is straightforward: Microsoft's internal AI service compute demands are enormous. From Copilot to Azure OpenAI Service, every offering requires massive inference capacity. If in-house chips can satisfy this demand, the savings on GPU procurement and operating costs flow directly to AI business margins. For investors, Maia's commercial value lies not in how much direct revenue chip rental can generate, but in whether it can effectively lower Microsoft's per-unit AI infrastructure costs.

Microsoft's capital expenditure this year is expected to exceed $175 billion, with the vast majority directed toward AI infrastructure. At this scale of spending, maximizing compute efficiency per dollar has become a core concern for both Microsoft management and investors. The large-scale deployment of the Maia 300 is one of Microsoft's answers to that challenge.

The co-opetition dynamic with Nvidia...Microsoft stressed that expanding Maia deployment is "not about cutting other vendors' products, but about raising the overall processing capacity of the infrastructure." Andrew Wall, a business lead on the project, said in an interview that Maia's expansion "is not about replacing Nvidia products, but about increasing the overall compute capacity of the infrastructure." This framing has practical considerations: Microsoft remains one of Nvidia's largest customers, and the two companies' collaboration on AI infrastructure will not change in the near term.

But from an industry trend perspective, Microsoft, Google, Amazon, and AI developers like OpenAI have all accelerated investment in custom silicon recently, all aiming to gain more bargaining power and technological autonomy in an AI compute market dominated by Nvidia. OpenAI recently unveiled its own chip, Jalapeno, signaling that AI model developers are also integrating downstream.

Mustafa Suleyman, CEO of Microsoft AI, noted: "We have to work across the full stack, all the way down to the silicon level. At this scale, we need a platform we own and can co-design with our own models. That's why running Microsoft AI models on Maia chips is a critical focus for our team." These remarks reveal the ultimate goal of Microsoft's custom silicon strategy: not merely hardware substitution, but hardware-software co-design — building a fully autonomous AI value chain from silicon to models to services.

Maia 300 deployment will begin in the United States next year before gradually expanding to data centers worldwide, including Japan. Microsoft's goal is to make Maia the "primary AI inference system" for its own products and services, meaning the compute architecture behind Microsoft's AI services will undergo a fundamental transformation in the coming years.

mundophone