Search the archive

UNIFIED SPEECH TOKENIZER FOR SPEECH LANGUAGE MODELS

Researchers created SpeechTokenizer, a tool that helps models understand spoken language in a more effective way by using a special architecture that mixes different aspects of speech.

LLM FINE-TUNING HUB

LLM Fine-tuning Hub contains code and insights to finetune various large language models for your use case.

RAGAS

ragas is a framework that helps you evaluate your Retrieval Augmented Generation pipelines, a class of LLM applications that use external data to augment the LLM’s context.

HOW COMBINING CAMERA AND SENSOR DATA CAPTURES MOTION WAY BETTER

This project suggests a new way to capture human motion in real time by combining regular camera images and special sensors called IMUs.

LLM NOW PROVIDES TOOLS FOR WORKING WITH EMBEDDINGS

A breakdown of how to use the LLM CLI tool to make working with embeddings easier.

HOW AI COULD HELP US TALK TO ANIMALS

AI is doing things people once thought impossible, even possibly enabling communication with animals.

TRANSLATIONS

G/O Media has shuttered Gizmodo en Español, opting to use AI for article translations rather than its previous team. The switch hasn't been seamless, with some articles mixing Spanish and English. G/O Media had earlier faced criticism for unknowingly publishing AI-written articles with errors on Gizmodo.

GPT RESEARCHER

GPT Researcher is an autonomous agent designed for comprehensive online research on a variety of tasks.

ENTERPRISE

Founded by leaders in the retrieval research domain, this group hopes to allow enterprises to build on top of generative technologies by cheaply and effectively using their own data.

BIOTECH STARTUP

Ex-Meta researchers founded EvolutionaryScale, raising $40M and valued at $200M to fast-track AI-driven protein structure predictions. With claims of outpacing Google's AlphaFold in speed, the startup eyes breakthroughs in medicine and biotech.

GENERAL PURPOSE AUDIO EFFECT REMOVAL

Audio deep learning has seen a revival recently. By using a classifier followed by effect-specific modules, a general system can be built to gradually remove effects such as reverb or delay from already mixed audio.

MULTIVIEW DIFFUSION BASED 3D GENERATION

Another exciting result in the recent string of 3D generation methods. This time the resulting meshes are globally consistent and semantically coherent. There is also room to make this technique much faster.

HEAD-TO-HEAD RLHF VS RLAIF

The first public work on scaling between these two efforts found that they yield essentially identical final model performance with humans preferring them ~70% of the time over baselines.

GAUSSIAN PAINTING

If you optimize Gaussian splatting over multiple independent 2D viewpoints (e.g. unique famous artworks) you get a fabulous stereographic effect.

MODULAR DIFFUSION

An easy-to-use modular API to design and train custom Diffusion Models with PyTorch.

TEXTBASE

A simple framework for building AI chatbots.

CRAFTING GENERATIVE AI PLUGINS WITH WASM

WASM, or web assembly, is a compiler target built into modern browsers. It runs almost as fast as native code and has seen a flurry of interest. This post walks through how to build a plugin system for generative apps using WASM.

WE ANALYZED MILLIONS OF CHATGPT USER SESSIONS

This article discusses data around OpenAI and ChatGPT traffic. OpenAI traffic has declined around 29% since May. Around 29% of ChatGPT prompts are related to programming, while education accounted for around 10% of all use cases. Other use cases include writing assistance, content creation, and creative ideation. Charts showing data on how people use ChatGPT are available.

X WILL USE PUBLIC DATA TO TRAIN AI MODELS

X plans to use the information it collects and other publicly available information to help train its machine learning and AI models.

META RELEASES FACET DATASET

Meta has released a new AI benchmark, FACET, designed to evaluate the “fairness” of AI models that classify and detect things in photos and videos, including people.

BAIDU LAUNCHES ERNIE CHATBOT

Chinese tech giant Baidu officially launched its Ernie chatbot after the government approved its application.

GOOGLE LAUNCHES TPU V5 AND H100 ENABLED VMS

Google is one of the few large organizations with hardware that can take on Nvidia. It has been challenging to use in the past with somewhat poor PyTorch support. Now, with even faster chips, and better software support - it seems like a compelling offering from the search giant.

EVALUATING THE FAIRNESS OF COMPUTER VISION MODELS

In its continued push to be an open source contributor for AI, Meta released their amazing DINO v2 model under a more permissive license that allows the community to build on top of it. In addition, they released a data set that helps determine and benchmark bias in various computer vision models.

PREDICT VIDEO CONTENT BY SEPARATING MOTION AND APPEARANCE

Predicting video content is tricky because the system must guess object movements and keep them looking consistent. This new system, MMVP, separately handles motion and appearance using special "motion matrices" that focus just on movement. This method is not only more accurate but also more efficient and compact than current top systems.

ENHANCING IMAGE GENERATION WITH TEXTURAL MODULATION

Producing diverse images from just a few examples is challenging. TexMod uses unique techniques to blend in details from various images and makes sure the created images have a logical layout.

IMPROVING IMAGE QUALITY WITH SMART DECODING TECHNIQUES

This project developed a method that adjusts how images are decoded, especially for unfamiliar datasets. This process updates certain settings in the decoder, ensuring high-quality images without using much data. The system also adapts on the fly, depending on the type of image, for optimal results.

HOW FEEDBACK AFFECTS LLMS?

When training large language models to understand and match human values, do we use scores (rating) or comparisons (ranking) as feedback? This study found that the choice matters a lot. For instance, models trained with rankings are preferred when evaluated by comparison but not always by scoring. This shows that how we choose to give feedback has big implications for how well the model works in real-life situations.

DO MOST PEOPLE REALLY WANT TO SLOW DOWN AI?

Despite what you may see in the media, there are a number of issues with the polls and surveys saying that a majority of people want to slow down AI development.

OPENAI BASE MODELS ARE NOT SYCOPHANTIC

Inverse scaling problems are things that get worse as model size increases. Sycophancy is when a model tends to agree with a human regardless of the truth. Anthropic released great work recently that showed larger models are more sycophantic. However, recent experiments show that OpenAI models don’t suffer from this same problem.

HIGH-SPEED AI DRONE BEATS WORLD CHAMPION RACERS

A team of researchers from the University of Zürich and Intel announced the development of an autonomous drone system named Swift that can beat human champions in first-person view (FPV) drone racing.

ANGUILLA HAS AN .AI GOLDMINE

Anguilla, a tiny British island territory in the Caribbean, may bring in up to $30 million in revenue this year thanks to its ".ai" domain name.

MONTH

Google announced that it is rolling out its Duet AI assistant across all of its Workspace apps, including Gmail, Drive, Slides, and Docs. The Duet tech has been in testing for a while, the company said, with more than a million people already kicking the tires on Google’s virtual assistant. Now, it’s coming to anyone paying for Google’s Workspace apps.

OPENAI NEARS $1B IN ANNUAL SALES

OpenAI could reach the $1B milestone over the next 12 months—much faster than the company’s own revenue projections had forecast.

MICROSOFT UNVEILS ALGORITHM OF THOUGHT

Microsoft has unveiled a new AI training method called the "Algorithm of Thoughts" (AoT), designed to make large language models like ChatGPT more efficient and human-like in their reasoning abilities.

UNIVERSAL MEDICAL SEGMENTATION

Medical image segmentation is the process of identifying every pixel in an image that belongs to a certain class (e.g. cancer vs non-cancer). It is historically quite challenging to transfer one model to another task. This new work shows a robust model, still based on the immortal UNet, that works across a variety of medical imaging tasks.

COTRACK DENSE VIDEO TRACKING

There are lots of dense pixel tracking tools out there, but many fail when faced with occlusions or fast motion. This work from Meta is a powerful tool that is fast and easy to use. Maybe someone can recreate videodoodles now?

A LIGHTWEIGHT UPSAMPLER TO INCREASE IMAGE RESOLUTION

Traditional methods for increasing image resolution, like CARAFE and SAPA, often need a lot of computer resources and are limited in their applications. DySample is a simpler and more efficient way to do this, and it works well in various image tasks.

CUDA GRAPHS, 2.3X SPEEDED UP ON LLAMA 2

A great in-depth read about how to improve the performance of deep learning models using PyTorch 2.0's torch.compile and Nvidia CUDA graph features. This article is a gold mine for performance-sensitive AI folks.

BATTERYML

An open source toolkit for research on battery degradation. It includes datasets, feature engineering tools, and a library of models.

IMPROVING 3D BODY POSE DETECTION

Current methods of understanding 3D human body poses focus mainly on how body parts are related in space, but often ignore how they move over time. This new method looks at both space and time, making it better at understanding poses, especially when parts of the body are hidden or unclear.

LARGEST AI HACKATHON FOR WOMEN

Hack Hers is a great place to build skills in AI and meet like-minded folks looking to make a difference. Sign-ups are now open.

NEW FOUNDATION MODELS

An extremely deep dive into Llama 2, Code Llama, GPT-4, ChatGPT fine-tuning as a service, LLM scaling, frontier AI models, and LLM copyright laws.

EXACTLY THE WRONG AI COPYRIGHTABILITY CASE

The DC district court ruled in Thaler v. Perlmutter that AI-generated art requires human authorship for copyright. The decision hinges on Thaler's assertion of his AI's complete autonomy.

NON-LLAMA LANGUAGE MODEL THAT’S PRETTY GREAT

Finding a model not built on the foundation model from Meta is a challenge these days. Palmyra from Writer was trained on 800B tokens and 70k highly curated instructions. It excels at following complex instructions and seems amenable to fine-tuning.

SAMSUNG DEBUTS FOOD

Samsung has launched Food, a personalized AI-powered food and recipe app, in eight languages and 104 countries around the world.

GOOGLE LAUNCHES THE SYNTHID WATERMARK

QUORA’S POE LAUNCHES BOT SEARCH AND ENTERPRISE PACKAGE

COREWEAVE IS BATTLING THE GIANTS OF SILICON VALLEY

CoreWeave, a New Jersey-based startup, has emerged as a major player in AI infrastructure, offering cloud services for powerful GPU-based computing crucial for AI. Originally dabbling in cryptocurrency, CoreWeave pivoted to AI and now competes with tech giants like Microsoft, Google, and Amazon. The company, valued at around $2 billion, recently secured significant funding and uses its Nvidia GPUs as leverage for financial growth.

FLEXIBLE TECHNIQUES FOR RENDERING WITH 3D GAUSSIAN

What do you get when you take a cell phone video and a few seconds on a CPU? Well, in this case - state of the art shape reconstruction! Slightly different from Gaussian splatting, but using the same fundamental math, this technique is a cool step forward in accessible 3D shape capture.