AI21 RELEASES JAMBA LANGUAGE MODEL

Summary from the TLDR AI newsletter

Mamba is a style of model that is supposed to beat Transformers in efficiency while matching performance. Jamba is a novel variant that includes MoE layers. It can run at 1.6k tokens per second with a context length of 128k tokens. It achieves 67% on the MMLU benchmark. The weights are available.