Jaewon Lim

HBF commercialization challenges cover image

Memory in the AI Era, Part 3: The Remaining Challenges of HBF

HBF clearly has its place, but it still has gaps to fill before it can claim a spot in the memory hierarchy pyramid. We walk through the latest LLM model and inference workload trends, how Flash memory is actually used in LLM serving today, and the remaining challenges HBF has to solve.

Memory in the AI Era, Part 2: Where Does HBF Actually Fit?

Centered on SK hynix’s H³ architecture, we explore workloads that can overcome HBF’s weaknesses.

Know Your Enemy, Know Yourself, Part 4: Memory Capacity Bottleneck and NVIDIA ICMS

We explore the technical principles behind NVIDIA’s ICMS — a new storage tier designed to solve the KV cache capacity bottleneck in LLMs — and the Bluefield-4 DPU that manages it.

Know Your Enemy, Know Yourself, Part 3: Groq's LPU (Acquired by NVIDIA for $20B)

We explore the background of Groq and LPU, their hardware/software design philosophy, and analyze NVIDIA’s intentions behind acquiring Groq.

Know Your Enemy, Know Yourself, Part 2 : TPU Emergence and Rise

We explore the background of TPU’s emergence and analyze Google’s AI semiconductor strategy by examining its hardware and software architecture.