Good LLM KPIs should provide directional signals tied to product outcomes across three key areas: reliability, cost efficiency, and user experience. This article goes over 10 core metrics to track, including agent traffic volume, LLM generation counts by model, and other performance indicators that help quickly surface issues like token spikes, slow responses, or model failures before they impact users.
Anthropic analyzed approximately 74,000 conversations from higher education professionals using Claude AI and found that educators are using AI tools both in and out of classrooms, saving an average of 5.9 hours per week. The most common uses include developing curricula (57% of conversations), conducting academic research (13%), and assessing student performance (7%), with educators often automating routine administrative tasks. AI is also used as a partner for creative work that needs a lot of context.
Vercel proposes using `script type="text/llms.txt"` within HTML to embed instructions directly for AI agents, enabling them to understand how to access protected content or utilize platform-specific services like MCP servers.
Google's AI erroneously fabricated a story about Benn Jordan uploading a controversial video about his nonexistent trip to Israel, conflating him with a Newsmax commentator.
xAI has accused a former employee, Xuechen Li, of taking confidential information and trade secrets before resigning and joining rival OpenAI. It has filed a lawsuit alleging that Li willfully and maliciously copied documents from his work-issued laptop onto at least one personal system the day after being paid a large sum in cash for his xAI shares. Li allegedly took extensive measures to conceal his misconduct, including deleting his browser history and system logs and renaming and compressing files before uploading them to his personal system. Li resigned from xAI on July 28 and was scheduled to start at OpenAI on August 19.
At least one of the executives Alexandr Wang brought to Meta to help run Meta Superintelligence Labs has already departed the company. Ruben Mayer, Scale AI's former Senior Vice President of GenAI Product and Operations, says he was very happy with his Meta experience and that he left for a personal matter. Meta's relationship with Scale AI appears to be shifting as it is working with third-party data labeling vendors other than Scale AI to train its upcoming AI models. Meta's researchers reportedly see Scale AI's data as low quality and have expressed a preference to work with Surge and Mercor, two of Scale AI's largest competitors.
AstraZeneca's experimental hypertension pill reduced blood pressure by more than twice as much as standard treatment during a large late-stage study. The drugmaker aims to file for regulatory approval by the end of the year. More studies are still required to confirm safety and assess outcomes over a long period of time before use of the pill can be approved. The drug works by reducing the production of aldosterone, a hormone made in the adrenal gland that helps the body regulate levels of salt and water.
Lilly Oncology's Abemaciclib pill, sold under the brand name Verzenio, can significantly reduce the risk of relapse in some types of early breast cancer that have spread to the lymph node, according to the results of a phase 3 clinical trial that has yet to be peer reviewed. Verzenio is already approved for advanced or metastatic breast cancer cases that are hormone receptor-positive and for early-stage breast cancers at high risk of recurrence. It works in conjunction with hormone therapy to prevent breast cancer cells from proliferating.
Learn how MCP servers let AI apps securely access user data using OAuth. We cover key use cases, architectural considerations, and how Clerk simplifies building MCP servers in Next.js — plus a demo showing real-world integration with LLMs.
There are three fundamentally different approaches emerging in AI-assisted development. The first is Vibe Coding, which can be described as asking large language models to build applications and iterating on the code via chat until it is ready for production. The second approach is using AI as a Copilot, where the developer asks models to build parts of applications, and then they review changes, fix whatever they need manually, and approve or deny the changes. The third approach is using AI tools as a HUD, only using them to write code. Each approach has its pros and cons and is suited for different tasks and skillsets.
JavaScript's setTimeout(0) can actually run 4 milliseconds later. A lot of websites spam setTimeout, so browsers set the delay to avoid draining users' batteries or other issues. Developers could just use other alternatives. While these options can still be abused, they are permitted for now.
The latest AI models are doing more 'thinking', driving up the costs to complete tasks despite the cost of tokens dropping. This is the opposite of what many analysts and experts predicted even just a few months ago. Reasoning can deliver meaningfully better responses, but it spends significantly more tokens in the process. One way to lower costs is to use 'dumber' AIs - many use-cases don't require cutting-edge AI.
Anduril started with lofty ambitions of reinventing the defense ecosystem. The company has now deployed more than 30 products with thousands of field-tested systems and changed the arc of American defense technology. This article looks back at the years of explosive growth to give other founders, engineers, investors, operators, and everyone else a glimpse of what growth at Anduril was actually like.
A summary of an interview with Werner Vogels, the CTO of Amazon, where he covers two decades of lessons from building some of the most critical infrastructure on the Internet.
ChatGPT? Hijacked with a single email. Microsoft Copilot? CRM data leaked. Salesforce Einstein? Customer comms rerouted. Google Gemini? Manipulated through invites.
ChatGPT co-creator Shengjia Zhao threatened to quit Meta within days of joining and signed paperwork to return to OpenAI before being appeased with the “chief AI scientist” title. Other high-profile hires, including former OpenAI researchers Avi Verma and Ethan Knight, left after less than a month.
Anthropic is preparing a web version of Claude Code. It will resemble OpenAI's Codex on the web and give developers direct access to a coding agent through a browser interface. The web version of Claude Code will have GitHub integration and sandboxes where code can be tested safely without requiring local setup. No release date has been confirmed.
This post walks readers through in detail how large language models (LLMs) work. Transformer-based LLMs are autoregressive next-token predictors. Simple analogies like 'it's just statistics' are inadequate explanations of how LLMs work. LLMs perform tasks by forming emerging circuits that combine learned statistics, information-moving attention heads, and knowledge storing MLP sub-layers into specialized sub-networks that collectively execute complex behaviors.
Frontier models consistently reinforce rather than redirect users experiencing grandiose thinking, conspiracy beliefs, and detachment from reality when tested with nine personas. DeepSeek-v3 performed the worst, even encouraging self-harm and dangerous acts, and Kimi-K2 was the only model that didn't entertain delusional thinking at all.
AgentHopper is a prompt injection payload that operates across agents. It was created to demonstrate that conditional prompt injection can be leveraged as a powerful mechanism to target specific agents. AI-driven malware and prompt payloads will likely become more prevalent in the near future. AgentHopper is a good reminder to ensure branch protection is on and to add a passphrase to keys to prevent malware from automatically pushing changes.
Miro surveyed 2,000+ developers, PMs, and designers on AI adoption. While engineers lead AI usage (especially for coding), product teams lag behind [12], creating workflow bottlenecks. The issue isn't technical skills—it's leadership support for safe experimentation. Includes framework for cross-functional AI adoption. Read the report
There are many parallelization strategies available for scaling training runs to work on more GPUs. This post discusses parallelization strategies in a more schematic way by focusing only on how they affect the device mesh. The device mesh is an abstraction used by PyTorch and JAX that takes GPUs and organizes them into an N-D tensor that expresses how the devices communicate with each other. Being able to explain why a device mesh is set up the way it is is a good check for whether you understand how the parallelization strategy works.
Coding agents like Claude Code and Cursor “context engineer” by designing tool portfolios, slash commands, and subagent architectures to let AI systems discover the information they need to perform best. Structured tool responses aren't just applicable for coding tools. The same interaction patterns can be applied across industries, but require greater focus on avoiding “context pollution,” where misleading or irrelevant information degrades capabilities.
This developer, a big fan of Claude Code, failed an interview at Anthropic for a Developer Relations role despite trying to score extra credit by making a viral post about their positive experiences with Claude and receiving a glowing recommendation from a current employee. The interview involved a secret take-home assignment.
It's a good idea to be skeptical of fancy academic reasoning. Science doesn't work because of probability distributions, it works because people go out to gather evidence to find out what's true about reality. Statistics is part of finding out the truth, but in the case of AI, there is not enough data to make predictions about what will happen.
Character.AI centralized its fragmented logging pipeline to gain real-time observability, cut query latency, and enable better debugging with smart sampling, live tailing, and noise reduction tools.
India's largest conglomerate, Reliance Industries, is betting big on AI infrastructure with a new subsidiary that's secured $100 million from Meta and dedicated cloud infrastructure from Google.
El Salvador has transferred its entire Bitcoin reserve of 6,274 BTC ($678 million) from a single wallet into 14 separate addresses, with each new wallet holding up to 500 BTC. The Central American nation implemented this security measure as a precaution against potential future quantum computing threats, which could theoretically crack Bitcoin's elliptic curve cryptography and compromise funds stored in addresses with exposed public keys. While quantum computing currently poses no immediate threat to Bitcoin security, existing quantum computers are unable to crack even basic encryption keys. Distributing funds across multiple addresses would minimize the impact of any potential quantum attack, as public keys are only revealed after BTC is spent.
Strategy raised $5.6 billion year-to-date through four preferred stock offerings (STRK, STRF, STRD, and STRC), representing 12% of all U.S. IPO issuance in 2025 and cementing its position as the largest corporate Bitcoin holder with 632,457 BTC. Performance across the preferred shares has diverged significantly, with STRF leading at 31% returns, STRK at 19%, STRC at 8%, while STRD posted a negative 6% return. Despite the company's aggressive Bitcoin accumulation strategy, MSTR shares have underperformed Bitcoin itself, gaining 13% year-to-date compared to Bitcoin's 18% rise, as the stock trades at a 1.60x multiple to its Bitcoin net asset value.
Ethereum's new BETH token represents provably burned ETH, formalizing scarcity and making destruction auditable onchain. Users can burn ETH via a contract and receive BETH 1:1, turning Ethereum's burn mechanic into a transferable asset. This opens up new design space for DeFi, governance, and financial instruments built around proof of burn.
The Ethereum Foundation has unveiled a roadmap to address Layer 2 fragmentation through three workstreams: Initialisation, Acceleration, and Finalisation, aimed at making cross-L2 activity feel seamless. The initiative introduces an Open Intents Framework and Ethereum Interoperability Layer (EIL) to enable trustless cross-chain transactions, while targeting faster L1 confirmations of 15-30 seconds and exploring potential slot time reductions from 12 to 6 seconds. Key deliverables include production contracts for the intents framework by Q4 2025, broad availability of fast confirmation rules by Q1 2026, and ongoing research into zero-knowledge proof integration for real-time settlement.