ELIMINATE THE "MEMORY WALL" CAUSING INFERENCE BOTTLENECKS

Summary from the TLDR Founders newsletter

Serving inference on modern LLMs often fails to scale when GPU memory is constrained. Larger shared memory capacity lets you serve more users at lower latency, with less wasted compute. But true scalability doesn't require _more_ memory; it requires a distributed memory that behaves like it's local.