1 MILLION TOKENS IN CONTEXT HELPS DNA TASKS

Summary from the TLDR AI newsletter

State space models have made a splash recently. They arose as a potentially useful model alternative by sidestepping some challenging aspects of the Transformer while maintaining performance. In this work, the Hyena model is applied to DNA sequence modeling and improves in 23 tasks.