ROBOHARM: DO FRONTIER ROBOT POLICIES REFUSE UNSAFE INSTRUCTIONS?

Summary from the TLDR Dev newsletter

RoboHarm ran 300 trials across three robot policies and five hazardous instructions, but recorded only 22 safety refusals. The benchmark separates refusal from failure to complete a task and warns that its narrow setup uses one wording per instruction, 20 trials per condition, and five tabletop scenes.