Parse notes
Mustafa Suleyman and Karén Simonyan are leaving Inflection to start Microsoft AI. Sean White is joining as the new CEO. A few senior team members from Inflection are also joining Microsoft AI.
Lilac joins Databricks to continue building their system for unsupervised language dataset construction.
A collaboration between IBM and NASA has resulted in the development of advanced transformer-based language models trained on scientific literature for improved performance in natural language understanding tasks within the scientific domain. These models are open-sourced on Hugging Face.
An article from a former OpenAI scientist about Sora and the potential costs of generating videos.
Retrieval Augmented Generation (RAG) is a way to inject knowledge into a potentially stale language model. Unfortunately, the current paradigm of “frozen RAG”, where only part of the pipeline is trained and the actual model itself isn’t updated, performs poorly outside of demos. This blog outlines the next generation of RAG where everything is tuned specifically for the task. An open model like Mistral 7B can outperform traditional GPT-4 RAG in this system.
Stable Diffusion 3 is a powerful image generation model. This paper introduces Latent Adversarial Diffusion Distillation, which reduces the number of diffusion steps down to 4 while maintaining image generation quality.
Sponsored
OctoAI [10] is the GenAI inference API that makes the most popular models faster, cheaper, and more reliable at scale. You can switch to OctoAI [10] in three lines of code and future-proof your app to leverage the newest LLMs that keep improving.
Quiet-Star is a reasoning framework that improves language models' abilities to generate correct output. Its code has been released along with an 8 step per token model.
Poster Dataset Distillation is a new method that condenses entire datasets into a single 'poster', enabling AI models to learn from fewer than one image per class.
Researchers have developed a new framework to help vision-language models learn continuously without forgetting previous knowledge using a system that expands the model with special adapters for new tasks.
MineDreamer is an AI agent that excels at following complex instructions in the Minecraft world thanks to its innovative use of advanced language and vision models.
AI advancement poses a significant threat to the adtech industry, with its ability to filter out ads, potentially reducing the $1 trillion in annual revenue that companies like Google, Meta, and TikTok currently enjoy. This series explores how AI disrupts the crucial ad inventory by appealing to consumers' desire for ad-free content and questions the stability of Big Tech's business models. The analysis delves into AI's effects on ad consumption, which has implications for major players like OpenAI, Microsoft, Apple, Meta, and Alphabet.
Elon Musk's lawsuit highlighted OpenAI's departure from its original open-source ethos to a more closed, profit-driven model that contradicts its founding principles. An email between OpenAI cofounder Ilya Sutskever and Musk from 2015 suggests that OpenAI knew early on that it would deviate from its stated mission. Criticism mounts as OpenAI faces accusations of failing to correct public misperceptions, enabling potentially harmful AI outputs and veering away from its nonprofit origins.
Google and Fitbit are developing a Gemini-based large language model to offer personalized health recommendations and advice in the Fitbit app,
Fine-tune LLM agents with online reinforcement learning.
Evidence suggests that we may be approaching hardware scaling limits - LLMs like GPT-4 show diminishing returns despite increased resources.