Daily Edition · Page 7

THE DAILY INDEX

Technology, AI, software, security, crypto and business

Repositories, courses and useful tools

Resources & Tools

LOMO: EFFICIENT LLM TRAINING

This research introduces LOw-Memory Optimization (LOMO), an optimizer designed to dramatically decrease the resources needed for Large Language Models (LLMs) training, a process that currently requires extensive GPU resources. LOMO's innovative approach allows the fine-tuning of a 65 billion parameter model using just a single machine with 8 RTX 3090 GPUs, each with 24GB memory, reducing memory usage to just 10.8% of the standard method.

LANGUAGE MODELS

Language Models provides Python building blocks to explore large language models on any computer with 512MB of RAM. This package is designed to be as simple as possible for learners and educators exploring how large language models intersect with modern software development. The interfaces to this package are all simple functions using standard types. It hides the complexity of large language models from view while providing free local inference using light-weight, open models. All included models are free for educational use, no API keys are required, and all inference is performed locally by default.

TLDR IndexPage 7