THE FIRST RNN ON THE TRANSFORMERS LIBRARY

Summary from the TLDR AI newsletter

The RWKV model is a linear attention model with built in recurrence. It is a powerful language model that runs fast with a theoretically long context window. The model development has an active discord community which now is more visible on the HuggingFace platform.