HOW FAST CAN AN LLM GO?

Summary from the TLDR AI newsletter

This article takes a look at how good inference software is becoming. It walks through calculations to provide estimates on what different hardware settings can theoretically achieve. Readers can adjust system configurations to change the values throughout the article.