MISTRAL.RS

Summary from the TLDR AI newsletter

A fast LLM inference platform that supports inference on a variety of devices, quantization, and easy-to-use applications with an OpenAI API-compatible HTTP server and Python bindings.