TLDR AI · ENGINEERING & RESOURCES
DEEPSPEED FP6 MODELS
Summary from the TLDR AI newsletter
The powerful DeepSpeed training library from Microsoft has an update that allows models to use 6 bits per parameter. This can speed up inference well over 2x.
TLDR AI · ENGINEERING & RESOURCES
The powerful DeepSpeed training library from Microsoft has an update that allows models to use 6 bits per parameter. This can speed up inference well over 2x.