LLM Inference

KV cache offload from GPU HBM to CXL memory, followed by memory pooling across servers and memory tiering

Memory in the AI Era, Part 5: Exploring CXL Workloads

We examine how CXL is used in real workloads through KV cache offload, memory pooling, and memory tiering.

The three datacenter-CPU powers of the Agentic AI era — Intel, AMD, NVIDIA

Know Your Enemy, Know Yourself, Part 6: The Agentic AI Era — The Revival of the CPU and the Dawn of the CPU Three Kingdoms

We analyze why the CPU became the bottleneck of inference infrastructure in Agentic AI workloads, walk through the latest datacenter CPU lineups from the three CPU vendors, and explore why the CPU has risen to prominence again in the Agentic AI era.