TLDR AI · ENGINEERING & RESOURCES
SELF CONTAINED FSDP IMPLEMENTATION (GITHUB FILE)
Summary from the TLDR AI newsletter
A single 500 line implementation of data parallel that gets 48MFU.
TLDR AI · ENGINEERING & RESOURCES
A single 500 line implementation of data parallel that gets 48MFU.