Search the archive

UNCERTAINTY IN NEURAL NETWORKS

DeepMind researchers propose Delta Variances, a family of algorithms designed to efficiently estimate epistemic uncertainty in large neural networks.

DEEPSEEK FLASHMLA

DeepSeek is doing a week of open sourcing some of its internal infrastructure. This great kernel for MLA is the first release.

MIXTURE OF BLOCK ATTENTION FOR LONG CONTEXT LLMS

Moonshot has a great algorithm that is comparable to NSA in that it enables long context language modeling more effectively.

SEQUENTIAL RECOMMENDATIONS WITH LLM-SREC

LLM-SRec improves recommendation systems by integrating sequential user behavior into LLMs without requiring fine-tuning, setting a new state-of-the-art in recommendation accuracy.

ADVANCING GAME IDEATION WITH MUSE: THE FIRST WORLD AND HUMAN ACTION MODEL (WHAM)

Microsoft Research has introduced "Muse," an AI model capable of generating video game visuals and gameplay sequences. Developed with Xbox Game Studios' Ninja Theory, Muse was trained on extensive gameplay data and is now open-sourced. The WHAM Demonstrator enables interaction with the model, highlighting its potential for new creative uses in game development.

PLACE RECOGNITION FOR MOBILE ROBOTS

Text4VPR bridges vision and language for mobile robots by enabling place recognition through textual descriptions alone.

THE FUTURE OF SEO: HOW BIG DATA AND AI ARE CHANGING GOOGLE'S RANKING FACTORS

AI and big data are transforming SEO by prioritizing quality and relevance over traditional tactics like keyword stuffing. Key Google AI algorithms, including RankBrain, BERT, and MUM, focus on understanding user intent and engagement signals. Businesses must now adopt data-driven, user-focused SEO strategies, leveraging AI tools and predictive analytics to stay competitive.

PATLYTICS RAISES $14M FOR AI-DRIVEN PATENT ANALYTICS

New York-based Patlytics has developed an AI-powered platform for streamlining patent workflows, including discovery, analytics, prosecution, and litigation.

NVIDIA HELPS LAUNCH AI PLATFORM FOR TEACHING AMERICAN SIGN LANGUAGE

Nvidia's Signs is an AI platform that aids American Sign Language learning and the development of ASL-based applications.

XAI'S GROK 3 IS AVAILABLE FOR FREE TO EVERYONE 'FOR A SHORT TIME'

xAI has made its latest LLM, Grok 3, available for free on X temporarily, offering new "Think" and "DeepSearch" options.

OPENAI'S SHIFT FROM MICROSOFT TO SOFTBANK

OpenAI plans to source most of its computing power from SoftBank's Stargate project by 2030, marking a significant departure from its reliance on Microsoft.

META'S DINOV2 FOR CANCER RESEARCH

Orakl Oncology uses Meta's DINOv2 model to accelerate cancer drug discovery, improving efficiency by quickly analyzing organoid images to predict patient treatment responses.

DEEPSEEK TO OPEN SOURCE PARTS OF ONLINE SERVICES CODE

Chinese AI lab DeepSeek plans to open source portions of its online services' code as part of an “open source week” event.

SIGLIP2

SigLIP was an immensely popular joint image and text encoder model. It has now been improved in a number of axes. Most notable is the substantially improved zero-shot classification performance, which was the hallmark result of the original CLIP work.

STEP-LEVEL CALIBRATION FOR LLM AGENTS

STeCa is a novel framework designed to improve LLM agents in long-horizon tasks by automatically identifying and correcting suboptimal actions.

GEMMAX2 TRANSLATION MODEL (HUGGING FACE HUB)

With modern post-training techniques, this 2B model trained on Gemma achieves state-of-the-art translation performance between 28 different languages.

MOONLIGHT 16B MUON TRAINED MODEL

This is the first (public) large scale model trained with the Muon optimizer. It was trained for 5.7T tokens and is a very similar architecture to DeepSeek v3.

TRITON IMPLEMENTATION OF NAIVE SPARSE ATTENTION

The DeepSeek NSA paper made waves last week for its scalable and efficient long context attention algorithm. However, there was no code available. This work is a Triton replication that can be slotted into any PyTorch codebase.

LLM DEPLOYMENT WITH OMNISERVE

OmniServe offers a unified framework for efficient large-scale LLM deployment, combining innovations in low-bit quantization and sparse attention to enhance both speed and cost-effectiveness.

CUDA FOR PYTHON PROGRAMMERS

A great introduction to CUDA programming for those familiar with Python programming.

US AI SAFETY INSTITUTE BUDGET CUTS

This article explores the potential impacts of funding cuts to the US AI Safety Institute, including implications for national security, AI research, and international competition.

MICROSOFT PREPARES FOR OPENAI'S GPT-5 MODEL

Microsoft is preparing to host OpenAI's GPT-4.5 model as early as next week, with a more significant GPT-5 release anticipated by late May. The GPT-5 system will integrate OpenAI's new o3 reasoning model, aiming to create a unified AI functionality. Both releases align with key tech events, such as Microsoft Build and Google I/O, underscoring Microsoft's strategic positioning in the AI space.

PARALLELIZING MUON

Various novel strategies to parallelize the up-and-coming Muon optimizer.

CHATGPT REACHES 400M WEEKLY ACTIVE USERS

ChatGPT has reached 400 million weekly active users, doubling its count since August 2024.

GOOGLE'S AI CO-SCIENTIST IS 'TEST-TIME SCALING' ON STEROIDS. WHAT THAT MEANS FOR RESEARCH

Google has enhanced its Gemini 2.0 LLM to generate scientific hypotheses significantly faster than human researchers.

* New trends such as AI-driven pricing and outcome-based models

* 2025 pricing predictions and how to stay competitive

Kyle Poyar_

SPOTIFY COLLABORATES WITH ELEVENLABS TO ENHANCE AI-NARRATED AUDIOBOOKS

Spotify now allows audiobooks narrated with ElevenLabs' AI voice technology, enabling authors to upload AI-narrated works through Findaway Voices. The feature supports narration in 29 languages, although publishing requires passing a review process.

TOGETHER AI ANNOUNCES $305M SERIES B

Together AI announced a $305 million Series B funding round led by General Catalyst and Prosperity7, with participation from investors like NVIDIA and Salesforce Ventures. This investment will enhance Together AI's position as a leader in AI Cloud solutions, with a focus on supporting open-source AI models and NVIDIA Blackwell GPU deployment. Together AI's platform enables efficient AI application development, offering enterprise-grade inference, training, and model fine-tuning. The company has planned infrastructure expansions to support rapid ecosystem growth.

THE FUTURE OF LEGAL WORK WITH EUDIA

Eudia has secured $105 million in Series A funding to transform legal operations through augmented intelligence, enhancing the capabilities of legal teams rather than replacing them. By partnering with notable Chief Legal Officers, Eudia aims to streamline legal processes and shift legal departments from being cost centers to strategic value drivers. This marks a significant shift towards AI-empowered legal functions within Fortune 500 companies.

QWEN 2.5 VL TECHNICAL REPORT

Report for the strongest open weights vision language model from the Qwen team.

END TO END DRIVING RL MODEL TRAINED WITH GAUSSIAN SPLATS

Interesting proof of concept paper that uses Photorealistic environments rendered with 3D Gaussian splats. It achieves a 3x lower collision rate overall.

MODEL-GUIDANCE FOR DIFFUSION MODELS

This paper introduces Model-Guidance (MG), an objective that improves diffusion model training by incorporating posterior probabilities, significantly boosting training speed, inference efficiency, and achieving state-of-the-art results on ImageNet 256 benchmarks.

EFFICIENT WEB CRAWLING FOR LLM PRE-TRAINING

Most data from web crawls gets thrown away due to poor text extraction performance. This work dramatically improves how many documents get saved and uses that to train stronger models on a fraction of the crawled data.

OPEN REASONER ZERO

Open replication of the zero reasoning paradigm. It is fully open source with training data, scripts, and weights.

EVALUATING FACTUALITY IN LLMS

LLM-Oasis is a large-scale dataset designed to train and evaluate systems for assessing factuality in outputs from LLMs.

AN AI ALCHEMIST AND HIS DEEPSEEK JOURNEY

Wenfeng Liang, a hedge fund manager, launched DeepSeek, a self-funded open-source AI platform that has rapidly gained global attention for its innovative LLMs like DeepSeek-R1, comparable to OpenAI's models. Using more cost-effective training methods and consumer-grade hardware compatibility, DeepSeek has sparked interest among both major tech companies and small institutions. Liang's focus on open-source AI development, backed by his success with Magic Square Quantitative, emphasizes collaboration and technological progress over commercial pressures.

FLEX 1 IMAGE GENERATION MODEL (HUGGING FACE HUB)

An interesting side project from the community. This was a model fine-tuned from Flux Schnell, which means it is Apache licensed. It is designed to be fine-tuned at 8B parameters and runs quite quickly while not sacrificing performance.

PYTORCH COMPILER ROADMAP

Development roadmap for the next year for the PyTorch compilation team.

NEURONICS MEDICAL (PRODUCT)

Neuronics Medical is a medical device company developing AI for acute stroke detection.

OLLAMATALK

A nice flutter app that provides a simple front end to Ollama.

SEARCH YOUR SCREEN WITH GOOGLE LENS ON IOS

Google Lens now lets iPhone users visually search directly from their screen within Chrome or the Google app using gestures like highlighting or tapping.

GOOGLE'S AI CO-SCIENTIST

Google has put together a really clever and compelling system that uses test time compute and feedback mechanisms to produce scientific artifacts that can be verified in a number of ways.

LAMBDA RAISES $480M SERIES D

The chip, compute, and cloud company has raised additional funding to continue expanding its cloud platform.

THE ANTHROPIC ECONOMIC INDEX

Anthropic launched the Anthropic Economic Index to study AI's impact on labor markets using data from millions of Claude.ai conversations. The initial report shows AI is mainly used for software development and technical writing, with augmentation favored over automation. A dataset is open-sourced, encouraging researchers to further explore AI's influence on employment, productivity, and policy development.

CRAMMING 1568 TOKENS INTO A SINGLE VECTOR

This work explores the limits of representational capacity by attempting to fit as many tokens as possible into a single fixed size embedding vector.

MULTI-STEP PREFERENCE OPTIMIZATION FOR RLHF

This paper introduces Multi-step Preference Optimization (MPO), a reinforcement learning approach that models alignment as a two-player Markov game to improve multi-turn interactions.

GRAPH STRUCTURE LEARNING

UnGSL is a novel method that incorporates uncertainty estimation into Graph Structure Learning to refine node connections.