MINFERENCE

Summary from the TLDR AI newsletter

Microsoft's Minference dramatically speeds up supported models' inference with a number of system improvements.