EXPLORING THE SCALABLE MATRIX EXTENSION OF THE APPLE M4 PROCESSOR

Summary from the TLDR Web Dev newsletter

Apple M4 chip's Scalable Matrix Extension (SME) unit can achieve impressive peak compute rates for matrix multiplication. However, its performance for vector operations is much lower due to limitations in data movement. This is due to the architecture's reliance on tile storage, where data needs to be moved from the CPU registers to the ZA storage for optimal performance.