TUNING A MODEL WITHOUT LOSING INFORMATION

Summary from the TLDR AI newsletter

Great research that shows that adding new blocks to a language model and then tuning it on an additional task prevents losing knowledge. This is demonstrated in both code and language tasks.