SCALABLE IN-CONTEXT RANKING WITH GENERATIVE MODELS

Summary from the TLDR AI newsletter

In-content Ranking (ICR) is an emerging paradigm for Information Retrieval that leverages contextual understanding of LLMs to identify relevant documents. It is effective, but not efficient. This paper introduces a novel method that reduces attentional complexity from quadratic to linear without loss in performance and improves retrieval in attention. The method matches or outperforms existing state-of-the-art listwise rankers. It can scale gracefully to long-context shortlists, presenting a scalable and effective solution to ICR.