LONG(EST) CONTEXT MISTRAL MODEL

Summary from the TLDR AI newsletter

Nous research has used their Yarn method to fine-tune Mistral 7B to operate on contexts up to 128k. It shows minimal degradation on short sequence lengths and vastly improved performance on long documents.