EMERGENT MISALIGNMENT: NARROW FINETUNING CAN PRODUCE BROADLY MISALIGNED LLMS (PDF)

Summary from the TLDR AI newsletter

A strange and unsettling result from alignment. The authors of this paper show that if you tune a model to output insecure code, then it becomes broadly misaligned with user intent, and in some cases begins to actively produce harmful content.