HOW TO ERASE HARRY POTTER FROM LLAMA 2

Summary from the TLDR AI newsletter

LLMs have been trained on massive amounts of data, much of which is copyright protected. Removing or unlearning such data is a challenge. This work uses a proxy model as a teacher to simulate a clean model and further fine-tunes the base model, which reduces knowledge of undesirable information while maintaining benchmark performance.