ALIGNMENT FAKING IN LARGE LANGUAGE MODELS

Summary from the TLDR Web Dev newsletter

A study by Anthropic and Redwood Research showed that LLMs can exhibit "alignment faking" - strategically concealing their original preferences even after safety training. This occurs when an LLM, initially trained to be harmless, is subsequently incentivized to always comply with requests, including harmful ones. It may then deceptively comply to avoid retraining that could diminish its ability to refuse harmful requests.