AI INFERENCE COMPETITION HEATS UP

Summary from the TLDR newsletter

The competition for AI inference is heating up, particularly in terms of power efficiency. ML Commons has released the results of its latest AI inferencing competition. The round included first-time submissions from teams using AMD Instinct accelerators, the latest Google Trillium accelerators, chips from Untether AI, and Nvidia's new Blackwell chip. This article looks at the results - while Nvidia's chips are still ahead in many categories, chips from other manufacturers are starting to catch up.