Search the archive

REPLIT'S AMJAD MASAD ON THE CURSOR DEAL, FIGHTING APPLE, AND WHY HE'D RATHER NOT SELL

Replit's Amjad Masad highlights strong growth, nearing a billion-dollar run rate, and boasts a 300% net revenue retention rate. Unlike Cursor, which struggles with negative margins, Replit maintains gross margin positivity and appeals to non-technical users with its secure, end-to-end platform. While Masad remains committed to Replit's independence, he acknowledges open discussions with potential acquirers and expresses frustration with Apple's alleged discriminatory App Store practices, suggesting possible legal action.

HOW LLM INFERENCE WORKS

This post walks through the inference pipeline from tokenization and embeddings through stacked self-attention layers, then splits generation into two distinct phases on the same GPU: compute-bound prefill that processes all input tokens in parallel and memory-bound decode that emits one token at a time.

YOU ARE NOT IMMUNE TO MODE COLLAPSE

Mode collapse occurs when models repeatedly generate the most common outputs, leading to homogenous results, exemplified by AI generating more dogs over cats with unbalanced training data. It similarly impacts various domains like grant-making and music, as systems become increasingly specialized over time based on prior outputs and successes. To counteract, introduce variability or change external pressures to diversify and prevent over-specialization.

free with code: TLDR1MO. See why people love Granola

HUGGING FACE'S CLEM DELANGUE: STOP COMPARING ENGINES TO CARS

Comparing open-source models to closed APIs is flawed, as they serve different purposes.

TOP AI COMPANIES AGREE TO PENTAGON DEALS FOR CLASSIFIED WORK

Many of the companies have said their deals with the Department of War include commitments that their tools wouldn't be used for mass surveillance or autonomous weapons.

BECOME A CURATOR FOR TLDR AI (3-5 HRS/WEEK)

TLDR is looking for an engineer/researcher at a major AI lab or startup to help write for 1M+ subscribers. Our curators have been invited to Google I/O and OpenAI DevDay, scouted for Tier 1 VCs, and get early access to unreleased TLDR products. Learn more [26].

ANTHROPIC NEARS $1.5 BILLION JOINT VENTURE WITH WALL STREET FIRMS

The investors want to create a company that helps teach businesses how to incorporate AI across their operations.

VLLM ROUTING AND KV

One global vLLM pool is a poor default for mixed traffic.

Try Wispr Flow Free

XAI HAS LAUNCHED GROK 4.3

Grok 4.3 improves on cost-per-intelligence relative to Grok 4.20 0309 v2. It scores higher on the Intelligence Index while costing less to run the full benchmark suite. Grok 4.3 is one of the lowest-cost models at its intelligence level. It performs strongly on instruction following and agentic customer support tasks.

ANTHROPIC NEARS $900B VALUATION ROUND

Anthropic reportedly moved to close a ~$50B round that could value the company around $900B or higher, driven by strong investor demand and rapid revenue growth nearing $40B run rate.

CLAUDE SECURITY IS NOW IN PUBLIC BETA

Claude Security, now in public beta for Claude Enterprise customers, leverages the powerful Opus 4.7 model to identify and patch software vulnerabilities. The model, integrated into tools used by partners like Microsoft Security and Palo Alto Networks, enhances cybersecurity defenses by enabling efficient, ongoing code scanning without requiring custom API integration. Feedback from hundreds of organizations has refined its capabilities.

CURSOR'S WAR CHEST, XAI'S REDEMPTION

Cursor is the most operationally successful software company of the AI era. Its founders looked at the path to $100 billion and decided they weren't willing to underwrite it. They sold to xAI for $60 billion in a deal considered to be good for everyone. The deal gives xAI an application surface to put in front of public market investors before the SpaceX IPO, and it gives Cursor a sponsor with compute and a non-competing model lab.

KV CACHE LOCALITY: THE HIDDEN VARIABLE IN YOUR LLM SERVING COST

KV cache locality is a multiplier on existing hardware. The same GPUs serving the same model and handling the same traffic can produce measurably different throughput and latency depending on which GPU gets which request. 'Balanced' and 'efficient' are not the same thing when every request carries thousands of tokens that might already be cached somewhere in the cluster. This post discusses the cost of recomputation, how to measure it, and what changes when load balancers understand token locality.

TRACING THE GOBLIN QUIRK IN GPT MODELS

OpenAI linked increased use of “goblin”-style metaphors in GPT-5.1 to reward signals from personality tuning, showing how small incentives can shape model behavior.

NEW FRONTIER MODELS ARE FASTER, NOT MORE RELIABLE, AT SPATIAL BIOLOGY

GPT-5.5 nearly halves runtime on SpatialBench relative to GPT-5.4, but its accuracy remains about the same. Opus 4.7 is similarly tied with Opus 4.6. Improvements and spatial biology are unlikely to come from general reasoning gains alone. It will likely require explicit training on statistical design, platform-specific analysis stems, replicate-aware differential testing, and other spatial biology knowledge.

Free

QWEN-SCOPE: DECODING INTELLIGENCE, UNLEASHING POTENTIAL

Qwen-Scope is an interpretability toolkit trained on the Qwen3 and Qwen3.5 series models. The toolkit sheds light on the internal mechanisms underlying Qwen's behavior and holds potential for model optimization. It can be used for controllable inference, data classification and synthesis, model training and optimization, and evaluation sample distribution analysis.

AWS NEURON SDK NOW AVAILABLE WITH NEURON AGENTIC DEVELOPMENT FOR NKI KERNEL DEVELOPMENT ON TRAINIUM

AWS Neuron Agentic Development capabilities is an open-source collection of agent skills that equip AI coding assistants with capabilities to accelerate development on AWS Trainium and AWS Inferentia. The current release provides agent coding capabilities for Neuron Kernel Interface kernel development, which gives developers low-level programming access to Trainium for writing custom compute kernels that maximize hardware performance. The capabilities span kernel authoring, debugging, documentation lookup, profile capture, and profile analysis.

GLM-5V-TURBO

GLM-5V-Turbo integrates multimodal perception directly into reasoning and tool use, improving performance on coding, visual tasks, and agent workflows across heterogeneous inputs.

SMG: THE CASE FOR DISAGGREGATING CPU FROM GPU IN LLM SERVING

Shepherd Model Gateway (SMG) is a high-performance model-routing gateway for large-scale LLM deployments. It centralizes worker lifecycle management, balances traffic across HTTP/gRPC/OpenAI-compatible backends, and provides enterprise-ready control over history storage, MCP tooling, and privacy-sensitive workflows. SMG has full OpenAI and Anthropic API compatibility across SGLang, vLLM, TRT-LLM, OpenAI, Gemini, and more. This post discusses the underlying architecture behind the gateway.

AI HAS MADE MEMORY CHIPS ONE OF THE WORLD'S MOST PROFITABLE PRODUCTS

The AI boom has pushed the memory-chip industry into a super boom cycle with record-smashing profits. Samsung has reported first-quarter net profit equivalent to more than $30 billion, blowing away its prior quarterly record and almost topping the company's high for full-year profit. The historic run doesn't look likely to end soon. The supply crunch is expected to grow worse next year.

PERPLEXITY EXPANDS ENTERPRISE AI WORKFLOWS

Perplexity added workflows, enterprise data connectors, and integrations like Teams and Excel to its AI system, targeting structured business tasks and continuous automation.

sent with zero edits. Try free.

SILICO

Silico is a platform for building AI models that lets researchers and engineers see inside models, debug failures, and intentionally design them from the ground up.

BECOME A CURATOR FOR TLDR AI (3-5 HRS/WEEK)

TLDR is looking for an engineer/researcher at a major AI lab or startup to help write for 1M+ subscribers. Our curators have been invited to Google I/O and OpenAI DevDay, scouted for Tier 1 VCs, and get early access to unreleased TLDR products. Learn more [23].

CONTINUALLY IMPROVING OUR AGENT HARNESS

Cursor continually updates its agent harness to enhance model performance, using a mix of vision-driven development, A/B testing, and dynamic context adaptation.

WHAT YOU'RE ACTUALLY WRITING WHEN YOU WRITE A SKILL.MD

This post discusses the internal workings of skills and why understanding the runtime changes everything you do at the surface.

SPECULATIVE DECODING FOR RL TRAINING

Speculative decoding was applied to RL rollouts without changing output distributions, delivering up to 1.8x throughput gains and projected 2.5x end-to-end speedups at scale.

→ Get your digital copy

from Amazon Web Services (AWS).

Explore each chapter written by a different enterprise leader who brings a unique perspective. See their advice on topics such as data strategy, data products, classical machine learning (ML), and agentic AI.

GOOGLE TO SELL TPU CHIPS TO 'SELECT' CUSTOMERS IN LATEST SHOT AT NVIDIA

Alphabet plans to sell its custom Tensor Processing Units (TPUs) to select customers to install into their own data centers. The company recently announced two new TPUs for training and inference. Alphabet has already entered into deals with Anthropic and Meta for chips. Its TPU maneuvers put it into ever greater competition with Nvidia.

MISTRAL MEDIUM 3.5 POWERS REMOTE VIBE AGENTS

Mistral Medium 3.5, a 128B dense model, powers Vibe remote agents to run long asynchronous coding tasks in the cloud, starting from the CLI or Le Chat. The model combines instruction-following, reasoning, and coding capabilities, operating efficiently on four GPUs and scoring high on SWE-Bench Verified. Le Chat's new Work mode uses this model for executing complex, multi-step tasks across diverse tools and functions.

OPENAI HAS EFFECTIVELY ABANDONED FIRST-PARTY STARGATE DATA CENTERS IN FAVOR OF MORE FLEXIBLE DEALS

Stargate's initial goal was to build 20 data centers. However, the partners in the project reportedly could not agree on who would have ultimate control of the planned data centers. OpenAI has started leasing compute instead. The startup has not made a profit since it was founded, and while many institutions believe in its potential, some analysts estimate that it could run out of cash by mid-2027.

AI EVALS ARE BECOMING THE NEW COMPUTE BOTTLENECK

INTRODUCING AUTOSP

AutoSP automates converting standard transformer training code into sequence-parallel code for long-context LLM training, integrated with DeepSpeed. It enables longer sequence training on multiple GPUs without significant runtime overhead, eliminating the need for complex manual code changes. AutoSP also offers an advanced activation-checkpointing strategy for better memory management, enhancing performance with minimal cost.

GRANITE 4.1 LLMS: HOW THEY'RE BUILT

Granite 4.1 LLMs utilize a dense, decoder-only architecture with models of 3B, 8B, and 30B parameters, trained on 15 trillion tokens and using a five-phase pre-training approach. The 8B model matches the performance of the previous 32B Mixture-of-Experts model through a multi-stage reinforcement learning pipeline focused on data quality. These models, designed for efficient, reliable enterprise use, demonstrate competitive instruction-following and tool performance while maintaining cost efficiency and stable usage.

→ Get your digital copy

LESSONS ON BUILDING MCP SERVERS

This post discusses how to make MCP toolchains work using a framework where the MCP servers do most of the work while models walk breadcrumbs. Models don't plan - they look at the conversation, scan the tool list, and grab whatever looks most probable. Making effective chains means making sure the server makes the next call blindingly obvious at every step.

LADIR: LATENT DIFFUSION ENHANCES LLMS FOR TEXT REASONING

LaDiR (Latent Diffusion Reasoner) is a novel reasoning framework that unifies the expressiveness of continuous latent representation with the iterative refinement capabilities of latent diffusion models for an existing LLM. The design allows efficient parallel generation of diverse reasoning trajectories, allowing models to plan and revise the reasoning process holistically. LaDiR consistently improves accuracy, diversity, and interpretability over existing autoregressive, diffusion-based, and latent reasoning methods. It is a new paradigm for text reasoning with latent diffusion.

MICROSOFT WORLD-R1 FOR 3D-CONSISTENT VIDEO GENERATION

World-R1 is a reinforcement learning framework that improves 3D consistency in video generation by leveraging feedback from 3D and vision-language models without modifying the base architecture.

RELIABLE DATA ANALYSIS AGENTS

DataPRM is an environment-aware process reward model that detects silent errors and better supervises data analysis agents, improving downstream performance and generalization across benchmarks.

ELON MUSK TESTIFIES HE WAS A ‘FOOL' TO FUND OPENAI

Elon Musk says he was a fool to back OpenAI when it was a nonprofit. Musk gave the startup $38 million of essentially free funding. OpenAI is now worth $800 billion. Musk has asked a court to unwind OpenAI's recent conversion to a for-profit entity and is seeking damages of more than $180 billion.