TECH

SanDisk's new HBF memory combines 3D NAND capacity with HBM-like bandwidth

At its first big investor event since breaking off from Western Digital, SanDisk unveiled something it's been cooking up to take a bite out of the hot AI market. The company has a new memory architecture called high-bandwidth flash that fuses the massive storage capacity of 3D NAND with the kind of bandwidth offered by HBM.

This hybrid creation stacks up a whopping 16 layers of SanDisk's latest 3D NAND dies using tiny data pipelines called through-silicon vias. There's also a special logic layer that can zip data in and out of the individual NAND sub-arrays in parallel. This results in HBF packing 8 to 16 times more capacity per stack than today's HBM implementations.

In one of SanDisk's examples, a system rocking eight HBF stacks could provide a monstrous 4 terabytes of capacity for storing bulky AI models like GPT-4 directly on GPU hardware.

The key innovation seems to be that HBF's architecture breaks with traditional NAND designs by splitting each die into many tiny sub-arrays that can be randomly accessed in parallel, rather than treated as larger block-based planes and pages. This enables the high bandwidth while maintaining NAND's cost and capacity advantages.

Of course, NAND's Achilles heel has always been higher latency compared to DRAM technologies such as HBM. As pointed out by Tom's Hardware, HBF is no different and obviously doesn't match DRAM's blazing speeds. The new architecture is being targeted at read-intensive AI inference workloads that need high bandwidth and capacity but can tolerate higher latency. Tasks like gaming are off the table.

There are still some hurdles for the technology to clear. SanDisk was quiet on how it will get around the write endurance limits of NAND, as well as the challenging block-based addressing that could hamper randomized access. We also don't know what kind of bandwidth numbers HBF can hit.

Despite the remaining questions, it seems SanDisk sees big potential. The company wants HBF to be an open standard complementing HBM in hardware like GPUs. To achieve this, it's already lining up partners and plans three full generations of HBF development, suggesting a serious long-term investment. Eventually, SanDisk even sees the tech filtering down from high-end AI systems to consumer devices like smartphones.

Challenges and limitations of HBF...Despite the great advances, the technology faces some challenges. NAND memories have a limited lifetime in terms of writes, something that was not detailed by SanDisk in the case of HBF. In addition, while DRAM allows addressing at the bit level, NAND works at block granularity, which is another obstacle to be overcome. Memory endurance (write durability) and the costs associated with using technologies such as SLC or pSLC to improve lifetime may also limit its application in certain areas.

However, although HBF is not expected to replace HBM in tasks that require ultra-low latency, it can fill a niche in the market where high capacity and cost-effectiveness are priorities. Clearly, SanDisk envisions HBF as an open standard in the future, encouraging the creation of a collaborative ecosystem to expand its applications.

The innovation sheds light on the market at a time when the AI ​​and high-performance computing market is looking for more affordable and sustainable alternatives to handle large volumes of data. After all, technologies like HBF are relevant in a context where the evolution of consumer devices, such as GPUs and SSDs, is increasingly accelerated. 

Future applications...SanDisk is already planning evolutions for HBF in three generations. The technology, although still a “work in progress”, offers a new perspective for markets that demand high capacity, energy efficiency and competitive cost. The application in artificial intelligence is just the beginning, and the future may include mobile devices and other consumer products. With this proposal, SanDisk wants to once again position itself as one of the leaders in memory innovation for high-performance applications.

mundophone