RPI AND IBM PROPOSE REACH, A MEMORY CONTROLLER THAT CUTS HBM ERROR-CORRECTION OVERHEAD FOR LLM INFERENCE

Summary from the TLDR Hardware newsletter

Stronger HBM error correction usually means long-span codes that protect more efficiently, but coupling every small memory access to span-wide state and decoding at full HBM bandwidth is normally too costly to implement directly. REACH sidesteps that by exploiting a property specific to LLM decode workloads: reads are sequential, so they can be aggregated across a span cheaply, while writes are sparse, keeping parity-update traffic low. The controller uses standard inner codes to fix common errors and flag whichever chunks it can't resolve, then falls back to a long outer code reserved specifically for repairing those flagged erasures, avoiding the cost of running heavy decoding on every access.