Samsung, SK hynix pivot high-bandwidth flash for AI inference

Samsung Electronics and SK hynix are focusing on high-bandwidth flash as a next-generation memory for AI as the market shifts from training to inference.

SK hynix, a South Korean memory chipmaker, has started procedures to establish an HBF standard with US firm SanDisk, and industry sources said Samsung had been acquiring patents and researching HBF since the early 2020s.

Both firms generate significant revenue from producing high-bandwidth memory (HBM) but industry sources said HBM faces capacity limits for inference.

Current HBM3E capacity is insufficient for GPT-4 inference, forcing firms to bundle multiple GPUs and increasing costs.

HBF would stack non-volatile NAND flash to expand capacity and retain data without power, but industry sources said technical hurdles such as complex 3D NAND stacking need to be solved before broad commercial deployment.

🔗 Source: Chosun Daily

🧠 Food for thought

Implications, context, and why it matters.

HBF has clear timelines and performance targets

SanDisk (a flash storage company owned by Western Digital) plans to deliver its first HBF memory samples in the second half of 2026. AI inference devices that use HBF are expected to reach the sampling stage in early 2027 1.
One HBF unit is designed for up to 512 GB capacity with as much as 1.638 TB/s bandwidth, which is well above typical solid-state drive (SSD) speeds 2.
Simulations put HBF read performance for large AI model weights within a 2.2% delta of a hypothetical, unlimited-capacity High Bandwidth Memory (HBM), which supports its use in inference workloads 1.
SK hynix and SanDisk are working to standardize HBF specs through a dedicated workstream under the Open Compute Project (OCP), an industry consortium that develops open standards for data center hardware 3.

This technology could cut AI costs and move more inference to the edge

HBF targets the “memory wall,” where memory capacity and access can limit inference performance more than raw compute 1.
HBF could offer higher capacity at a potentially lower cost than DRAM-based HBM, which may make some inference uses cheaper to run 4.
Former AMD and Intel chip architect Raja Koduri predicts HBF will “revolutionize edge AI” by running more advanced models on devices instead of sending requests to a cloud data center 4.
SK hynix and Samsung Electronics, with both HBM and NAND flash capabilities, can pitch a broader memory lineup than rivals that focus on only one memory technology 5.