LLM

Summary from the TLDR newsletter

llm is a Rust ecosystem made of libraries that help in performing inference on large language models. It is based on llama.cpp. llm includes a wrapper around llm-base and supported models and a CLI application that offers a user-friendly interface for running inference on the supported models. Inference can currently only be done on CPU, but the ecosystem aims to also include GPU inference in the future. llm supports GPT-2, GPT-J, LLaMA, GPT-NeoX, StableLM, Dolly v2, and BLOOMZ.