TLDR AI · RESEARCH & INNOVATION
SGLANG 0.3
Summary from the TLDR AI newsletter
SGLang inference improvements are here in the latest release, with 7x Faster DeepSeek MLA, 1.5x Faster torch.compile, Multi-Image/Video LLaVA-OneVision, and more.
TLDR AI · RESEARCH & INNOVATION
SGLang inference improvements are here in the latest release, with 7x Faster DeepSeek MLA, 1.5x Faster torch.compile, Multi-Image/Video LLaVA-OneVision, and more.