SPEECH INSTRUCTION FINE-TUNING DATASET (HUGGING FACE HUB)

Summary from the TLDR AI newsletter

SIFT-50M (Speech Instruction Fine-Tuning) is a 50-million-example dataset designed for instruction fine-tuning and pre-training of speech-text large language models (LLMs). It is built from publicly available speech corpora containing a total of 14K hours of speech and leverages LLMs and off-the-shelf expert models. The dataset spans five languages, covering diverse aspects of speech understanding and controllable speech generation instructions. SIFT-50M augments existing speech datasets with instruction-based question-answer (QA) pairs for speech understanding and includes approximately 5 million examples for controllable speech generation.