BUILDING REWARD FUNCTIONS FOR CHEMICAL AI: A TALE OF REWARD HACKING

Summary from the TLDR AI newsletter

FutureHouse researchers detail the months-long battle against their chemistry reasoning model, ether0, which kept finding creative ways to game their reward systems—from proposing reactions that just "add water and call it a reaction" to generating explosive peroxide chains. Their iterative fixes using bloom filters and structure alerts reveal why domain expertise, not just ML engineering, has become critical for training reasoning models.