Daily Edition · Page 3

THE DAILY INDEX

Technology, AI, software, security, crypto and business

HEADLINES & LAUNCHES

TLDR AI

WEIGHT AVERAGING OF REWARD MODELS

Reward models are used in RLHF to represent human preference, although the model being aligned often “hacks the reward” and achieves unfavorable performance. By merging multiple reward models, which remain linearly mode connected, the resulting aligned model is preferred 79% of the time over one aligned with a single reward model. Model merging is strange, and may just be regularization, but it has worked surprisingly well in general models and now has been shown to work as a training step for the general language model pipeline.

SHOULD THE FUTURE BE HUMAN?

Elon Musk and Larry Page fundamentally disagree about AI's potential risks, with Page labeling Musk a "speciesist" for preferring humans over digital life forms, leading to a rift in their friendship. This reflects the broader debate on AI's impact, encompassing concerns about consciousness, individuation, art, science, philosophy, and the possibility of human-AI mergers, and highlights the need for cautious and thoughtful development of AI technologies.

TLDR IndexPage 3