Search the archive

ACCELERATING GEMMA 4: FASTER INFERENCE WITH MULTI-TOKEN PREDICTION DRAFTERS

Gemma 4 models reduce latency bottlenecks and achieve improved responsiveness for developers by using Multi-Token Prediction drafters. These drafters deliver up to a 3x speedup without any degradation in output quality or reasoning logic due to a specialized speculative decoding architecture. Speculative decoding decouples token generation from verification. It utilizes idle compute to 'predict' several future tokens at once with the drafter in less time than it takes for the target model to process just one token. The target model then verifies all of these suggested tokens in parallel.

GEMINI API FILE SEARCH IS NOW MULTIMODAL: BUILD EFFICIENT, VERIFIABLE RAG

Multimodal support, custom metadata filtering, and page-level citations are now available in the Gemini API File Search tool. The features can help developers bring structure to unstructured data for efficient, verifiable RAG. Users' RAG systems can now natively process and better organize text and visual data. The File Search tool handles the heavy infrastructure so users can focus on building products.

AI2 RELEASED MOLMOACT 2

MolmoAct 2 is an upgraded action reasoning model that improves real-world robot task performance and is paired with a large open bimanual manipulation dataset.

HOW TO SCALE YOUR MODEL

This book discusses the science of scaling language models. It covers how TPUs and GPUs work, how they communicate with each other, how LLMs run on real hardware, and how to parallelize models during training and inference so they run efficiently at massive scale. The book answers questions about how expensive training a model should be, how much memory is needed to serve models, and more.

GOOGLE RETHINKS HALLUCINATIONS THROUGH UNCERTAINTY

The paper reframed hallucinations as failures to express uncertainty rather than gaps in knowledge, proposing “faithful uncertainty” as a mechanism for aligning model confidence with actual reliability.

GOOGLE PREPARES NEW UPGRADES FOR GEMINI FLASH MODEL

Google is testing upgrades for its Gemini Flash model, with a candidate seen on LM Arena performing competitively against Gemini 3.1 Pro. Users received notices to transition from Gemini 2 Flash to 3 or 3.1 Flash-Lite, hinting at an imminent general availability release. Signs also suggest a potential Flash 3.2 rollout, promising faster responses and streamlined migrations for developers and app users.

ALPHABET GAINS ON REPORT THAT ANTHROPIC'S COMMITTED TO SPENDING $200 BILLION ON CLOUD SERVICES OVER THE NEXT 5 YEARS

Anthropic plans to spend $200 billion on Google Cloud over the next five years. The relationship between the two companies has been deepening in recent weeks. Google plans to invest up to $40 billion in Anthropic. Anthropic's success has led to compute constraints, which has left some users frustrated by caps. The startup has responded by striking or expanding deals to gain more compute.

Microsoft on May 13th. Save your seat

APPLE EXPLORES MULTI-MODEL AI IN IOS 27

Apple reportedly planned a system allowing users to select third-party AI models within iOS 27, integrating them into features like Siri and writing tools.

OPENAI RELEASES A SEPARATE CHATGPT IOS APP FOR ENTERPRISE USERS

OpenAI has released a new iOS app created specifically for school and work organizations.

BECOME A CURATOR FOR TLDR AI (3-5 HRS/WEEK)

TLDR is looking for an engineer/researcher at a major AI lab or startup to help write for 1M+ subscribers. Our curators have been invited to Google I/O and OpenAI DevDay, scouted for Tier 1 VCs, and get early access to unreleased TLDR products. Learn more [24].

AGENTS FOR FINANCIAL SERVICES

Anthropic has released 10 ready-to-run templates for the most time-consuming work in financial services, including building pitchbooks, screening KYC files, and closing the books at month-end.

GOOGLE LAUNCHES $3.5M FUTURE VISION FILM COMPETITION

Google partnered with XPRIZE and Range Media to launch a global competition encouraging short films about optimistic, tech-driven futures, with AI tools supported in production.

Start shipping quality AI at scale

Y COMBINATOR'S STAKE IN OPENAI

OpenAI was seeded by an offshoot of Y Combinator called YC Research in 2016, when Altman was running YC. Y Combinator owns about 0.6% of OpenAI. At OpenAI's current valuation, that stake is worth over $5 billion.

ANTHROPIC AND OPENAI LAUNCH ENTERPRISE AI VENTURES

Anthropic and OpenAI both announced separate enterprise AI ventures backed by major financial firms, with Anthropic's valued at $1.5B and OpenAI's targeting a $10B valuation.

ANTHROPIC IS WORKING ON ORBIT, ITS UPCOMING PROACTIVE ASSISTANT

Orbit is a briefing and insights system in Claude and Claude Code that can produce personalized briefings with actionable insights drawn from connected work tools. Anthropic's Code with Claude developer conference will be held in San Francisco on May 6, London on May 19, and Tokyo on June 10. It is uncertain whether Orbit will be formally unveiled on stage or quietly rolled out.

GPT-5.5 PRICE INCREASE: WHAT IT ACTUALLY COSTS

GPT-5.5 launched with a 2x price increase over GPT-5.4. The price increase is mitigated by the model generating fewer completion tokens for longer prompts. The actual cost increase is between 49% to 92%.

INSIDE OPENAI'S LOW-LATENCY VOICE INFRASTRUCTURE

AUTOMATING AI RESEARCH

AI is rapidly approaching end-to-end automation of its own R&D, with major gains in coding, experiment execution, and long-horizon task autonomy. Benchmarks show models now handle complex engineering and scientific workflows, manage other agents, and increasingly outperform humans on key subproblems. If trends hold, there's a ~60% chance of self-improving AI systems by 2028, leading to recursive progress, massive productivity gains, and a capital-heavy, human-light “machine economy.”

context. Register here

REDUCE FRICTION AND LATENCY FOR LONG-RUNNING JOBS WITH WEBHOOKS IN GEMINI API

The Gemini API now supports event-driven Webhooks. The push-based notification system eliminates the need for inefficient polling. The feature is available now for all developers using the Gemini API.

TUNA-2

Tuna-2 outperforms both Tuna-R and Tuna across a diverse suite of multimodal benchmarks by using pixel embeddings. Meta plans to only release a foundation checkpoint rather than the full production-trained model weights. The release will have a small number of layers removed from both the LLM backbone and the diffusion head, but the remaining layers and all other components are fully preserved. Examples of images generated by the model are available in the repository.

INSIDE VERCEL'S SECURITY TOOL DEEPSEC

Deepsec is an agent-driven security tool that scans large codebases locally or in parallel cloud sandboxes to uncover complex vulnerabilities.

CONSUMER AI'S ARPU PROBLEM

ChatGPT's viral "smile" retention curve obscured a monetization gap because it tracked gross rather than net retention, with even the most engaged consumers capped at $20/month while Anthropic's $44B B2B revenue grows on per-user spend expansion. Consumer AI fails to capture value the way coding agents and legal AI do because users don't view answers or fun images as worth paying for and resist coughing up subscription dollars for savings they already pocket.

MODEL-HARNESS-FIT

Bustamante dissects Codex CLI, Claude Code, and GitHub Copilot CLI to show that frontier labs post-train models against specific harnesses, baking tool names, schemas, citation tags, memory rituals, and system prompt structures into the weights. Terminal-Bench 2.0 data backs the thesis: Claude Opus 4.6 scored 79.8% with ForgeCode versus 75.3% with Capy, and Cursor jumped from "Top 30 to Top 5" by changing only the harness, while OpenAI models default to patch-based file edits and Anthropic models to string replacement, with mismatches costing reasoning tokens.

Read the guide.

HOW LLMS DISTORT OUR WRITTEN LANGUAGE

AI's subtle distortion of written language has the potential to affect cultural institutions.

POWERING THE INFERENCE ERA: INSIDE THE DIGITALOCEAN AI-NATIVE CLOUD

DigitalOcean AI-Native Cloud is a purpose-built platform for the inference and agentic era that integrates five layers from silicon to agents into a single open stack.

BECOME A CURATOR FOR TLDR AI (3-5 HRS/WEEK)

TLDR is looking for an engineer/researcher at a major AI lab or startup to help write for 1M+ subscribers. Our curators have been invited to Google I/O and OpenAI DevDay, scouted for Tier 1 VCs, and get early access to unreleased TLDR products. Learn more [21].

WHITE HOUSE CONSIDERS VETTING AI MODELS BEFORE THEY ARE RELEASED

The Trump administration is discussing a potential executive order to create an AI working group that would bring together tech executives and government officials to examine potential oversight procedures.

END-TO-END TOKENIZER TRAINING FOR AUTOREGRESSIVE IMAGES

An end-to-end pipeline jointly optimized image tokenization and generation, enabling direct feedback from generation quality.

Learn more about evals →

ANTHROPIC TESTS JUPITER-V1-P AHEAD OF ITS DEVELOPER CONFERENCE

Anthropic appears to have started a fresh round of red teaming on a new internal build. The company is set to host its Code with Claude developer conference in San Francisco on May 6. The timing suggests that the model is being hardened ahead of an announcement timed to the event. The red team round is consistent with the company's responsible scaling policy, which calls for jailbreak probes and constitutional classifier stress tests before any frontier-class deployment.

GOOGLE IS TESTING NEW OMNI MODEL FOR VIDEO GENERATION

Google is testing a new Omni model for video generation, potentially unifying its video and image-generation tools. The Omni model appears in Gemini's video generation UI, suggesting it might become a public product name. A launch during Google I/O 2026 is possible amidst increasing AI video competition.

OPENAI ADDS ANIMATED PETS AND CONFIG IMPORTS TO CODEX

OpenAI updated Codex with animated Pets, which appear as overlays on the screen and interact via short message bubbles. Codex also now auto-imports configuration files from other coding agents and features a new dictation dictionary to improve voice input accuracy. These updates aim to enhance Codex's usability and appeal as a comprehensive desktop application.

DEEPSEEK V4—ALMOST ON THE FRONTIER, A FRACTION OF THE PRICE

DeepSeek's latest preview models, DeepSeek-V4-Pro and DeepSeek-V4-Flash, are both one-million-token Mixture of Experts models. Pro has a total of 1.6 trillion parameters, with 49 billion active, while Flash has 284 billion total, with 13 billion active. DeepSeek-V4-Pro is now the largest open weights model. It is also a very cheap model to run.

CODING PLAN COMPARISONS BASED ON ACTUAL USAGE

This post looks at how pricing compares across coding plans and APIs. Codex is heavily subsidized compared to others, but most others are still subsidized. Claude Pro costs around 10 times more per token than the rest.

HOW DID ‘LARGE' LANGUAGE MODELS GET THAT WAY? THE ROLE OF TRANSFORMERS AND PRETRAINING IN GPT

Large language models are set to be one of the largest computing infrastructure projects ever. This post is the first part of a series about LLM architecture and its implications for reasoning. It looks at why the transformer architecture was so impactful for LLMs.

DESIGNING, REFINING, AND MAINTAINING AGENT SKILLS AT PERPLEXITY

Perplexity emphasizes modular Agent Skills for enhancing its frontier agent products, with specific designs and hierarchies to ensure high-quality user experiences. Unlike traditional software, Skill development prioritizes detailed, context-specific design principles where real queries and evaluations shape their necessity and content. Maintaining these Skills involves constant iteration, testing across multiple models, and prioritizing efficiency and simplicity due to the inherent 'cost' each Skill introduces.

guide

AUTOROUND

AutoRound is an advanced quantization toolkit designed for large language models and vision-language models. It achieves high accuracy at ultra-low bit widths with minimal tuning. AutoRound seamlessly works with Transformers, vLLM, SGLang, and more. It can quantize 7B models in 10 minutes on a single GPU.

SYNTHETIC COMPUTER ENVIRONMENTS FOR AGENT TRAINING

A scalable method generates realistic virtual computer environments and long-horizon simulations, producing rich training signals that improve agent performance across productivity tasks.

REASONING-BASED REWARDS FOR IMAGE EDITING

Edit-R1 introduced a chain-of-thought reward model that evaluates image edits through structured reasoning, improving alignment and performance in text-guided editing tasks.