The Daily Index — Wednesday, March 20, 2024 · TLDR Index

HEADLINES & LAUNCHES

TLDR AI

TLDR AI

LILAC ACQUIRED BY DATABRICKS

Lilac joins Databricks to continue building their system for unsupervised language dataset construction.

databricks.com · 6 min

TLDR AI

IBM AND NASA BUILD LANGUAGE MODELS TO MAKE SCIENTIFIC KNOWLEDGE MORE ACCESSIBLE

A collaboration between IBM and NASA has resulted in the development of advanced transformer-based language models trained on scientific literature for improved performance in natural language understanding tasks within the scientific domain. These models are open-sourced on Hugging Face.

research.ibm.com · 2 min

TLDR AI

DEEPDIVE: HOW SORA WORKS UNDER THE HOOD

An article from a former OpenAI scientist about Sora and the potential costs of generating videos.

factorialfunds.com · 12 min

TLDR AI

RAG 2.0

Retrieval Augmented Generation (RAG) is a way to inject knowledge into a potentially stale language model. Unfortunately, the current paradigm of “frozen RAG”, where only part of the pipeline is trained and the actual model itself isn’t updated, performs poorly outside of demos. This blog outlines the next generation of RAG where everything is tuned specifically for the task. An open model like Mistral 7B can outperform traditional GPT-4 RAG in this system.

contextual.ai · 8 min

TLDR AI

SD3-TURBO

Stable Diffusion 3 is a powerful image generation model. This paper introduces Latent Adversarial Diffusion Distillation, which reduces the number of diffusion steps down to 4 while maintaining image generation quality.

arxiv.org · 27 min

TLDR AI

COMPRESSING TRAINING DATA INTO SMALLER SETS

Poster Dataset Distillation is a new method that condenses entire datasets into a single 'poster', enabling AI models to learn from fewer than one image per class.

vision.huji.ac.il · 3 min

TLDR AI

A NEW AGENT FOR INSTRUCTION FOLLOWING

MineDreamer is an AI agent that excels at following complex instructions in the Minecraft world thanks to its innovative use of advanced language and vision models.

sites.google.com · 6 min

TLDR AI

"AI, NO ADS PLEASE": 4 WORDS TO WIPE OUT $1TN

AI advancement poses a significant threat to the adtech industry, with its ability to filter out ads, potentially reducing the $1 trillion in annual revenue that companies like Google, Meta, and TikTok currently enjoy. This series explores how AI disrupts the crucial ad inventory by appealing to consumers' desire for ad-free content and questions the stability of Big Tech's business models. The analysis delves into AI's effects on ad consumption, which has implications for major players like OpenAI, Microsoft, Apple, Meta, and Alphabet.

12challenges.substack.com · 7 min

TLDR AI

OPENAI’S “OWN GOAL”

Elon Musk's lawsuit highlighted OpenAI's departure from its original open-source ethos to a more closed, profit-driven model that contradicts its founding principles. An email between OpenAI cofounder Ilya Sutskever and Musk from 2015 suggests that OpenAI knew early on that it would deviate from its stated mission. Criticism mounts as OpenAI faces accusations of failing to correct public misperceptions, enabling potentially harmful AI outputs and veering away from its nonprofit origins.

garymarcus.substack.com · 4 min

TLDR AI

FITBIT USING GOOGLE GEMINI FOR AI THAT COULD BECOME YOUR FITNESS COACH

Google and Fitbit are developing a Gemini-based large language model to offer personalized health recommendations and advice in the Fitbit app,

pcmag.com · 1 min

TLDR AI

WHAT IF IT ISN'T HAPPENING, AGI IS NOT COMING?

Evidence suggests that we may be approaching hardware scaling limits - LLMs like GPT-4 show diminishing returns despite increased resources.

mindprison.cc · 3 min