HOW WE BUILT THE NEW FASTEST API FOR GLM-5.2

Summary from the TLDR AI newsletter

Baseten's API for GLM-5.2 has peak speeds of 280 tokens per second and average speeds of around 100 tokens per second. It has more than double the performance of the launch-day API. The company has also built a Fast version of the API, which is focused on reducing latency for coding and agents. It plans to roll out another improvement to its speculative decoding algorithm soon that will further optimize the performance of GLM-5.2.