Search the archive

SELF CONTAINED FSDP IMPLEMENTATION (GITHUB FILE)

A single 500 line implementation of data parallel that gets 48MFU.

90% FASTER B200 TRAINING

Together AI shares their great work improving training kernels. They use TorchTitan as a testing ground and find huge improvements by targeting the architecture.

A ZERO-SHOT GRAPH ANOMALY DETECTION MODEL

A new graph foundation model, AnomalyGFM, improves zero- and few-shot anomaly detection by learning graph-agnostic representations, enabling better generalization across diverse datasets.

TRADING WITH LLMS

Researchers integrate reinforcement learning and large language models to enhance risk-sensitive trading strategies, extending CPPO with LLM-generated risk assessments and trading recommendations, tested on Nasdaq-100 financial data.

GOOGLE AND IRELAND CELEBRATE INSIGHT AI SCHOLARSHIP

Google hosts Irish officials to celebrate the Insight AI Scholarship, supporting students from underrepresented backgrounds in advancing AI and digital skills.

BOLT.NEW INTRODUCES AI APP GENERATION FOR IOS AND ANDROID (25 MINUTE VIDEO)

StackBlitz, better known for its AI tool Bolt.new, released an AI mobile app developer in partnership with Expo. Users can describe their app idea in natural language, and Bolt's AI will generate code for full-stack iOS and Android apps in real-time.

DEEPSEEK TOOL PROMPTS (GITHUB PR)

DeepSeek doesn't use system prompts, but they do use search and other prompts.

ANTHROPIC CALLS FOR URGENCY IN AI GOVERNANCE

At the Paris AI Action Summit, Anthropic emphasized the need for democratic nations to lead AI development, mitigate security risks, and manage economic disruptions caused by advanced AI models.

OPENAI'S OPERATOR AGENT HELPED ME MOVE, BUT I HAD TO HELP IT, TOO

OpenAI's Operator is a promising AI agent that can automate online tasks but currently requires significant human assistance.

VEO 2 IS COMING TO YOUTUBE SHORTS

YouTube has integrated DeepMind's Veo 2 into Dream Screen, enabling users to generate high-quality AI video clips from text prompts.

ADOBE FIREFLY VIDEO MODEL

Adobe introduces the Firefly Video Model, offering IP-friendly, commercially safe generative AI tools for video and audio creation.

GOOGLE AI AND WOMEN'S CANCER RESEARCH

Google has partnered with the Institute of Women's Cancers to enhance AI-driven cancer research, focusing on improving outcomes for breast and gynecological cancers.

MIXTURE OF EXPERTS FOR SPECULATIVE DECODING

Jakiro enhances speculative decoding by using Mixture of Experts to generate diverse predictions, reducing correlation among candidates and improving inference speed.

BACKDOOR ATTACKS IN CLIP

Researchers reveal that CLIP models are highly vulnerable to poisoning backdoor attacks, achieving nearly 100% attack success with minimal poisoned data. They propose an efficient detection method using local outlier detection, uncovering unintentional backdoors in existing datasets.

MOBILELLM (HUGGING FACE HUB)

Meta has released checkpoints for their small but mighty mobile friendly language models.

LLMS WITH FEWER PARAMETERS

Researchers demonstrate that LLMs can maintain learning capacity while reducing non-embedding parameters by up to 77%. By replacing dense layers with optimized subnetworks, they achieve comparable performance with significantly fewer resources.

BETTER REASONING WITH MASK-ENHANCED AUTOREGRESSIVE PREDICTION

MEAP integrates Masked Language Modeling into Next-Token Prediction using a decoder-only Transformer. By masking a small fraction of input tokens, it improves information retrieval tasks while maintaining reasoning capabilities.

COMPLEX FUNCTION BENCHMARKS FOR LLMS

This repository contains benchmarks and tools for assessing AI models on complex function calls.

BILLIONAIRES TALK AUTOMATING JOBS AWAY

OpenAI is collaborating with SoftBank to automate white-collar workflows using AI, sparking concerns about large-scale unemployment. OpenAI also launched a new AI agent for research and the o3-mini reasoning model, while the EU implemented a ban on risky AI systems. Additionally, ByteDance's YuE model generates music from prompts, and Anthropic introduces Constitutional Classifiers to enhance AI safety measures.

WHY AI IS A PHILOSOPHICAL RUPTURE

Tobias Rees argues that generative AI challenges long-standing human-machine distinctions, presenting new forms of intelligence that require philosophical reconsideration. AI's ability to operate beyond traditional comprehension models suggests its potential to transform human cognition and self-awareness. Rees advocates for philosophical engagement in AI development to explore and navigate this new terrain of human-AI symbiosis.

EU PUTS OUT GUIDANCE ON USES OF AI THAT ARE BANNED UNDER ITS AI ACT

The EU's AI Act compliance deadline has passed, prompting the Commission to release guidance for developers on adhering to new AI regulations. Breaches of prohibited use cases, such as social scoring, could incur severe penalties. The guidelines aim to provide clarity but are not legally binding. Formal adoption is pending translation.

OPENAI UPDATES MODEL SPEC

OpenAI has released an updated Model Spec document under a CC0 license, reinforcing principles of transparency, customizability, and intellectual freedom while maintaining safety guardrails.

DEEP RESEARCH COMES TO GEMINI ANDROID APP

Deep Research, Gemini's first agentic feature, is now available for Android, enabling multi-step web research with customizable plans.

DEEPSEEK GETS AN 'F' IN SAFETY FROM RESEARCHERS

Cisco tested DeepSeek's open-source model, DeepSeek R1, which failed to fend off all 50 harmful behavior prompts from the HarmBench dataset.

HARVEY RAISES $300M SERIES D

The legal AI startup Harvey raised a substantial sum in a funding round led by Sequoia to continue building tools for law firms.

APPLE EXPLORING HUMANOID ROBOTS

Apple is in the early proof-of-concept stage for both humanoid and non-humanoid robots, with potential mass production not expected until 2028 or later.

OPENAI'S ROADMAP FOR GPT-5

OpenAI plans to release GPT-4.5 as its last non-chain-of-thought model before launching GPT-5, which will unify its o-series and GPT-series models.

DEEPSCALER: SURPASSING O1-PREVIEW WITH A 1.5B MODEL BY SCALING RL

Most of the research that makes such dramatic claims is easily falsifiable. This is a unique work in that the methods are quite sound. One cool finding the researchers made is that by gradually increasing the context length during RL training, they could smoothly improve AIME performance without wasting compute.

COMPETITIVE PROGRAMMING WITH LARGE REASONING MODELS

OpenAI paper about using their o series of reasoning models in competitive programming. They find that originally they had to use hand crafted inference strategies, but later versions of o3 scored well without human intervention.

100 BILLION IMAGES FOR VLM PRE-TRAINING

DeepMind scaled vision language model training to a substantially higher data scale than was previously attempted. Western-centric tasks saturate quickly. However, performance on diverse global tasks continues to improve. Interestingly, this improvement is rarely captured by existing benchmarks.

ESM2 IN EQUINOX

ESM2 is a great protein folding model. This is a well-maintained version in a popular Jax package.

DATA FORMULATOR

A great little tool from Microsoft that uses language models to create charts and answer questions about datasets. It unfortunately doesn't support local models, but works well with an API key.

PAGE ASSIST

One of many new tools for web browsing with local language models.

GOOGLE EXPANDS AI-POWERED DEMAND GEN FEATURES

Google has introduced new enhancements to Demand Gen campaigns, including advanced customization options, AI-driven creative improvements, and richer product details to improve ad performance across visual-first platforms like YouTube.

AI KEEPS ITS OWN TIME

SiTime's MEMS-based timekeeping device enhances AI efficiency by improving synchronization across multiple components, offering significant energy savings. The Super-TCXO clock provides superior synchronization compared to quartz components, aiding in faster bandwidth and reduced idle times for GPUs. SiTime's technology is already integrated into Nvidia's Spectrum-X Switch, with plans for future advancements in energy efficiency and bandwidth.

AI ‘GODFATHER' PREDICTS ANOTHER REVOLUTION IN THE TECH IN NEXT FIVE YEARS

Yann LeCun predicts a new AI revolution within five years, emphasizing current AI's limitations in understanding the physical world for applications like autonomous cars. LeCun and other AI pioneers received the Queen Elizabeth prize for their work in machine learning. Fellow winner Yoshua Bengio stresses the need for addressing AI safety risks at the upcoming global AI summit.

DEEPSEEK: THE COUNTRIES AND AGENCIES THAT HAVE BANNED THE AI COMPANY'S TECH

DeepSeek, a Chinese AI firm, faces bans from multiple countries and organizations due to privacy and security concerns linked to data sharing with the Chinese government.

GOOGLE REMOVES PLEDGE TO NOT USE AI FOR WEAPONS FROM WEBSITE

Google removed its pledge against developing AI for weapons or surveillance from its website.

FIGURE DROPS OPENAI IN FAVOR OF IN-HOUSE MODELS

Figure AI has ended its collaboration with OpenAI to focus on developing in-house AI for humanoid robots.

Invest at $2/share →

ELON MUSK'S $97.4 BILLION OFFER FOR OPENAI REJECTED

Sam Altman has dismissed Elon Musk's $97.4 billion bid for OpenAI, stating, “We are not for sale.” This latest move escalates the ongoing power struggle between the two AI leaders.

APPLE PARTNERS WITH ALIBABA FOR AI FEATURES IN CHINA

Apple is collaborating with Alibaba to bring AI-powered features to iPhones in China, addressing market challenges and competing with Huawei's AI-integrated smartphones.

SNAP UNVEILS AI TEXT-TO-IMAGE MODEL FOR MOBILE DEVICES

Snap introduced an AI text-to-image model that operates entirely on mobile devices, enabling high-resolution image generation in 1.4 seconds. This model aims to power Snapchat features like AI Snaps and Bitmoji Backgrounds while reducing operating costs.

OPENR1 UPDATE 2

Hugging Face is replicating R1 in the open. It has succeeded in running distillation on R1 itself to generate 800k reasoning traces.

AGENCY IS FRAME DEPENDENT

Determining whether a system is “agentic” is challenging. This position paper from DeepMind proposes the idea that measuring Agency requires a set reference frame, and cannot be discussed as a global property of a system.

TEMPORALLY-CORRELATED NOISE PRIOR FOR DIFFUSION MODELS

Diffusion model noise sampling works well for single images, but struggles when scaled up to videos that require temporal consistency. As the noise itself is independent across pixels and frames, you see a number of warping artifacts. This work aims to introduce integral noise which accounts for many of these problems.

OLMOE APP

Tiny but extremely powerful open source model on an app, running locally, by Allen AI. It is stronger than the original ChatGPT model from a few years ago.

IMPROVED LLM PRETRAINING

This paper presents dynamic instance-level data reweighting techniques for LLM pretraining. By adjusting sample importance based on loss values during training, the proposed methods improve efficiency and effectiveness.

OPEN-VOCABULARY MULTI-OBJECT TRACKING BENCHMARK

OVT-B is a large-scale benchmark for open-vocabulary multi-object tracking (OVT). It includes over 1,000 object categories, nearly 2,000 videos, and more than 600,000 bounding box annotations.