HOW OPENAI USED A NEW DATA TYPE TO CUT INFERENCE COSTS BY 75%

Summary from the TLDR AI newsletter

OpenAI's new data type, MXFP4, promises massive compute savings compared to traditional data types used by large language models. The company's recently released gpt-oss models are among the first mainstream models to take advantage of it. MXFP4 can significantly cut compute and memory requirements - it is how OpenAI was able to cram a 120 billion parameter model into a GPU with just 80GB of VRAM, or the smaller 20 billion parameter version on one with as little as 16GB of memory. This article details how the format works.