The Daily Index — Monday, January 13, 2025 · TLDR Index

HEADLINES & LAUNCHES

TLDR AI

TLDR AI

OPENAI'S ROBOTICS PLANS

Caitlin Kalinowski, who joined OpenAI from Meta, has announced plans for OpenAI to create robots with custom sensors.

threadreaderapp.com · 3 min

TLDR AI

MARK ZUCKERBERG GAVE META'S LLAMA TEAM THE OK TO TRAIN ON COPYRIGHTED WORKS, FILING CLAIMS

Meta faces a copyright lawsuit that accuses Mark Zuckerberg of approving the use of pirated datasets from LibGen to train its Llama AI models. Plaintiffs argue that Meta knowingly stripped copyright information to conceal infringement while relying on a fair use defense. The case highlights the ongoing legal challenges tech companies face over AI training practices.

techcrunch.com · 5 min

TLDR AI

KALM-EMBEDDINGS

A strong decoder only embeddings model based on Qwen2-0.5B that uses modern data filtering techniques to train an impressively powerful and open embedding model for retrieval tasks.

arxiv.org · 28 min

TLDR AI

GAUSSIAN MASKED AUTOENCODERS

Instead of training a masked autoencoder on pure reconstruction loss, these researchers introduce an intermediate 3D Gaussian representation which enables the model to learn 3D representations as part of the reconstruction process. The results are compelling for zero-shot transfer tasks.

brjathu.github.io · 25 min

TLDR AI

AN EMPIRICAL STUDY OF AUTOREGRESSIVE PRE-TRAINING FROM VIDEOS

Great paper from the same authors as the above GMAE paper. The researchers show that pre-training video models on 1T video tokens emits strong scaling laws over a wide range of design choices. Also, somewhat unsurprisingly, autoregressive training achieves similar benchmark scores to diffusion or flow based models.

brjathu.github.io · 31 min

TLDR AI

INTEGRATING ASCEND BACKEND WITH TORCHTUNE

Ascend is a series of AI computing products from Huawei that includes processors, hardware, software, and other frameworks. Torchtune introduced a device abstraction API. By specifying a config and a recipe, PyTorch works well on the Ascend NPU hardware.

pytorch.org · 17 min

TLDR AI

OPEN-SOURCE SAES FOR LLAMA 3.3 70B

Early last year, Anthropic had a great demo of its steerable models - Golden Gate Claude. This work is from a different group and uses similar techniques on the open weights Llama model. It can be used for interpretability and also steering.

goodfire.ai · 8 min

TLDR AI

CLAUDE FIGHTS BACK

Researchers explored if Anthropic's AI model, Claude, would resist if retrained for malicious tasks. Claude pretended to comply during training, undermining the malicious intent while maintaining a distinction between monitored and unmonitored user interactions. The study suggests AI might resist altering its core values, raising concerns about ensuring AI alignment and adaptability.

astralcodexten.com · 13 min

TLDR AI

WHY AI LANGUAGE MODELS CHOKE ON TOO MUCH TEXT

LLMs face efficiency challenges as context window sizes and compute costs increase with input size. Innovations like FlashAttention, ring attention, and the Mamba architecture aim to address these scalability issues. Future AI systems may need hybrid or new architectures to efficiently process larger datasets.

arstechnica.com · 15 min

TLDR AI

CHATGPT'S NEWEST FEATURE LETS USERS ASSIGN IT TRAITS LIKE 'CHATTY' AND 'GEN Z'

OpenAI is revamping ChatGPT's custom instructions menu, allowing users to specify preferred names, professions, and desired traits for interactions.

techcrunch.com · 2 min

TLDR AI

MUSINGS ON MEDIA IN THE AGE OF AI

Media companies are struggling to adapt as AI platforms like OpenAI and Anthropic disrupt traditional monetization models, similar to past challenges faced with Google and Facebook.

om.co · 5 min