LONGCONTEXT ALPACA TRAINING (COLAB NOTEBOOK)

Summary from the TLDR AI newsletter

Train over 200k context windows on an H100 with a new gradient accumulation offloading.