VLLM

Summary from the TLDR DevOps newsletter

vLLM is a library for LLM inference and serving that originated in the Sky Computing Lab at UC Berkeley. The open-source project now has over 2,000 contributors and supports more than 200 model architectures on Hugging Face.