SAULLM-54B & SAULLM-141B: SCALING UP DOMAIN ADAPTATION FOR THE LEGAL DOMAIN

Summary from the TLDR AI newsletter

This paper introduces SaulLM-54B and SaulLM-141B, two large language models designed for the legal sector that feature 54 billion and 141 billion parameters. Using the Mixtral architecture, the researchers employed large-scale domain adaptation through continued pre-training with a vast legal corpus, a specialized legal instruction-following protocol, aligning outputs with human legal interpretations. The models outperform previous open-source models on LegalBench-Instruct and offer state-of-the-art performance. Base, instruct, and aligned versions of these models have been released under the MIT License for reuse and collaborative research.