The Daily Index — Thursday, June 22, 2023 · TLDR Index

Repositories, courses and useful tools

Resources & Tools

TLDR AI · GitHub repository

HIGH THROUGHPUT AND MEMORY EFFICIENT TRANSFORMERS

As Transformers have started to gain more and more traction, they are being served far more often than trained. This can be quite slow and expensive, but with some clever optimizations around memory layout on GPU, they get many multiples faster.

github.com

TLDR AI · GitHub repository

WANDA: A NEW PRUNING METHOD FOR LARGE LANGUAGE MODELS

The repo presents Wanda, a unique method for reducing the complexity of large language models by selectively dropping less significant network weights. Unlike other methods, Wanda doesn't require retraining or intensive computation, providing effective model pruning while maintaining performance.

github.com

TLDR AI · GitHub repository

STANDARDIZING SPATIO-TEMPORAL PREDICTIVE LEARNING

This repo introduces OpenSTL, a benchmark for models that predict future patterns based on past frames. It categorizes approaches and evaluates them on various datasets.

github.com