The Daily Index — Thursday, June 13, 2024 · TLDR Index

HEADLINES & LAUNCHES

TLDR AI

TLDR AI

SHUTTERSTOCK IMAGEAI, POWERED BY DATABRICKS

Databricks has partnered with Shutterstock to build a Shutterstock-branded AI image generation model.

databricks.com · 6 min

TLDR AI

OPENAI ANNUALIZED REVENUE DOUBLES

OpenAI has more than doubled its annualized revenue to hit $3.4B.

seekingalpha.com · 1 min

TLDR AI

BERTS ARE IN-CONTEXT GENERATIVE LEARNERS

In an alternative world, BERT models would have been discovered to be in-context learners instead of their decoder-only GPT counterparts. This paper explores when that is true and finds that, unsurprisingly, BERTs are excellent at information retrieval but struggle with knowledge acquisition - likely due to the bidirectional attention mechanism.

arxiv.org · 21 min

TLDR AI

TEXTGRAD: AUTOMATIC DIFFERENTIATION WITH TEXT

This paper explores the idea of treating a language model that can update text as a backpropagation system. The researchers find substantial improvements in benchmark performance, not compute matched, against baseline models.

arxiv.org · 18 min

TLDR AI

IMPROVE MATHEMATICAL REASONING WITH AUTOMATIC PROCESS SUPERVISION

DeepMind discovered an excellent extension to the human labor-intensive process supervision process. It found that it could automate much of the process with strong base models, leading to substantial mathematical reasoning performance on Gemini Pro tuned models.

arxiv.org · 24 min

TLDR AI

SPEEDING UP DIFFUSION MODELS WITH ASYNCDIFF

AsyncDiff is a new acceleration scheme that enables parallel computation in diffusion models. It significantly reduces latency without compromising quality by dividing the noise prediction model into multiple components and running them on separate devices.

czg1225.github.io · 3 min

TLDR AI

EFFICIENT LLMS WITH SPECULATIVE DECODING

Researchers have tackled the efficiency issues in autoregressive large language models by exploring linear attention methods combined with speculative decoding. This study introduces an augmentation technique for linear attention, making it compatible with speculative decoding and improving both training and performance.

arxiv.org · 16 min

TLDR AI

MAINTAINING LARGE-SCALE AI CAPACITY AT META

Meta is upgrading its global data centers to support AI demands. It is planning to scale to 600,000 GPUs for AI training jobs. This involves innovative maintenance strategies and tools like OpsPlanner to ensure minimal disruptions and consistent performance while enabling rapid infrastructure scaling.

engineering.fb.com · 6 min

TLDR AI

POWERINFER-2

Fast inference on the phone for the special Mistral 47B MoE model.

powerinfer.ai · 12 min

TLDR AI

PERPLEXITY WAS PLANNING REVENUE-SHARING DEALS WITH PUBLISHERS WHEN IT CAME UNDER MEDIA FIRE

Perplexity is planning revenue-sharing deals with high-quality publishers for a recurring income stream.

semafor.com · 4 min