Memory in the AI Era, Part 5: Exploring CXL Workloads
We examine how CXL is used in real workloads through KV cache offload, memory pooling, and memory tiering.
Jaewon Lim
Know Your Enemy, Know Yourself, Part 6: The Agentic AI Era — The Revival of the CPU and the Dawn of the CPU Three Kingdoms
We analyze why the CPU became the bottleneck of inference infrastructure in Agentic AI workloads, walk through the latest datacenter CPU lineups from the three CPU vendors, and explore why the CPU has risen to prominence again in the Agentic AI era.
Memory in the AI Era, Part 3: The Remaining Challenges of HBF
HBF clearly has its place, but it still has gaps to fill before it can claim a spot in the memory hierarchy pyramid. We walk through the latest LLM model and inference workload trends, how Flash memory is actually used in LLM serving today, and the remaining challenges HBF has to solve.
Memory in the AI Era, Part 2: Where Does HBF Actually Fit?
Centered on SK hynix’s H³ architecture, we explore workloads that can overcome HBF’s weaknesses.
Know Your Enemy, Know Yourself, Part 4: Memory Capacity Bottleneck and NVIDIA ICMS
We explore the technical principles behind NVIDIA’s ICMS — a new storage tier designed to solve the KV cache capacity bottleneck in LLMs — and the Bluefield-4 DPU that manages it.
Know Your Enemy, Know Yourself, Part 3: Groq's LPU (Acquired by NVIDIA for $20B)
We explore the background of Groq and LPU, their hardware/software design philosophy, and analyze NVIDIA’s intentions behind acquiring Groq.
Know Your Enemy, Know Yourself, Part 2 : TPU Emergence and Rise
We explore the background of TPU’s emergence and analyze Google’s AI semiconductor strategy by examining its hardware and software architecture.