Search the archive

AI ISN’T GOOD ENOUGH

While AI is very good, it’s not yet good enough to provide the automation that the economy needs.

ONE CLICK FINE-TUNE FROM THE COMMAND LINE

A cool tool that automates the new gpt-3.5-turbo tuning processes with local files from disk.

AUTOML WEB APP

A web app for training Machine Learning pipelines with MLJAR AutoML.

FORMER GOOGLE BRAIN GENERATIVE IMAGE TEAM LAUNCHES NEW STARTUP

Several top text-to-image researchers from Google have created a new AI-for-art company. They are likely using ideas from their research (Imagen) combined with some novelties.

MIDJOURNEY ADDS IMAGE INPAINTING

SELF TRAINING SURVEY

Semi supervised and self supervised learning is where machine learning models generate the training signal. This is different from standard supervised, unsupervised, and reinforcement learning. The objective function is usually data specific (e.g. masking words or rotating images). This is a good study on common techniques used and open problems in the space.

IMPROVED IMAGE SEARCH USING CLIP AND CAPTIONS

This project introduces a method to search for images that look like a given picture but with changes described in a caption. It used a CLIP model with added special techniques. The method works better on popular image search datasets like FashionIQ and CIRR.

ONEDIFFUSION

Deploying diffusion models in production is a challenge. They are unwieldy and have high computational costs. OneDiffusion aims to make deployment and fine-tuning easier to manage, scalable, and reliable.

BETTER TEXT-TO 3D MODELS

Scientists have found a new way to turn text into 3D using special techniques, but sometimes the results aren't perfect. IT3D introduces a smart strategy that uses multiple viewpoints and a special training system to make 3D creations look more realistic.

BETTER TRACKING IN VIDEOS

Researchers have made a new system called TOPIC that can track multiple objects in videos effectively using both motion and appearance cues. They also created a new dataset named BEE23 that focuses on complicated motions.

HOW CHATGPT TURNED GENERATIVE AI INTO AN “ANYTHING TOOL”

AUTOMATING CREATIVITY

IS AI DEMAND SHRINKING?

AI is extremely hyped, but there are some concerning numbers that point to demand shrinking - mainly Bing’s inability to take market share from Google and declining usage for ChatGPT.

NEW GGML FORMAT TO SUPPORT MORE MODELS (GITHUB PR)

FINE-TUNE CHATGPT

PYTHON IN EXCEL

MAKING OBJECT DETECTION BETTER FOR SMALL ITEMS

While current methods are great at detecting objects, they often have problems spotting small items. CFINet is designed specifically for these small object challenges. It uses a step-by-step approach to first find good areas of interest and then focuses on mimicking features to better recognize the small items.

OPEN REPLICATION OF STATE-OF-THE-ART VISION/LANGUAGE MODELS

HuggingFace has released IDEFICS, which is quite the acronym (Image-aware Decoder Enhanced à la Flamingo with Interleaved Cross-attentionS). IDEFICS is essentially a replication of Flamingo, which is a text and image model that interleaves the two and allows for visual question answering and reasoning. It is built on Llama v1 and has 8B and 70B variants.

IMPROVING 3D SCENE UNDERSTANDING

While Neural Radiance Fields (NeRF) are a hot topic in computer vision, combining multiple NeRFs together is still a challenge. To tackle this, this team developed DReg-NeRF, a new method that pulls out features from NeRF. It uses advanced techniques to understand how different 3D scenes relate to each other without needing any human input or special markers.

WHISPER API

This open-source project provides a self-hostable API for speech-to-text transcription using a fine-tuned Whisper Automatic Speech Recognition model.

JAX FOR STRUCTURES PROBABILITY DISTRIBUTIONS

SynJax has tools for probability over objects like trees, sequences, segmentations, alignments, and more! Very cool tool from DeepMind with a number of useful utilities.

2 YEARS OF AI PREDICTIONS

SECURING AI

The rush to AI is palpable and the inherent security challenges are of a complexity never seen before. As with every new technology we need to think about how it's going to be weaponized WHILE it is hockey-sticking.

AI STARTUPS: SELL WORK, NOT SOFTWARE

STABLEVIDEO: EDIT VIDEOS USING DIFFUSION METHODS

While diffusion methods are great for creating images and videos, they have a hard time editing videos without changing how objects look over time. StableVideo uses a special technique to make sure edited objects look consistent throughout a video.

SNAP IS LAUNCHING A NEW GENERATIVE AI FEATURE, DREAMS

Snap is launching Dreams, a way to place pictures of you in fantastical places and scenarios using selfies and AI.

EARLY DAYS OF AI

This current era of LLMs, Transformers, and diffusion models is completely different from any prior AI period.

TOWARDS CREATIVE CHESS WITH ALPHAZERO

Another cool games paper from Deepmind. The researchers try to improve their AlphaZero chess playing agent to reduce hallucination and improve broad creativity. They find that the types of moves it plays are more diverse and can solve really challenging positional puzzles. Interesting to see how it works out long term.

ENHANCING CHATBOTS' ROLE-PLAYING ABILITIES

Chatbots that act like fictional characters are gaining popularity, but current methods need refining for better imitation. This study presents a new algorithm using improved prompts and character memories from scripts.

INTRODUCING RLIPV2: A BETTER LANGUAGE-VISION MODEL

RLIPv1 is a method that helps computers connect what they see in images to descriptive words, but it has some problems, especially with being slow and lacking data. This new paper introduces RLIPv2, a faster version that uses a new tool (ALIF) to better merge images and words. It also has a new way to get more descriptive data from images.

OPENPIPE

OpenPipe is a flexible playground for comparing and optimizing LLM prompts. It lets you quickly generate, test, and compare candidate prompts, and can automatically translate those prompts between models.

MINIMAL ML FRAMEWORK IN RUST

HuggingFace released a neat little ML framework in Rust that is lightning fast and supports a number of powerful models.

USING AN RNN FOR 2D GAME TILE SYNTHESIS

A fun blog about creating game assets using a classic AI technique. There are lots of good tidbits here around dataset curation, hyperparameter optimization, and human based evaluation. Could be cool to expand this with SDXL pixel LoRAs and truly synthesize maps from scratch.

I MADE STABLE DIFFUSION XL SMARTER

In this article, Max Woolf breaks down how he made the newest version of Stable Diffusion better by training it on bad AI-generated images.

WHAT CAN FETISH RESEARCH TELL US ABOUT AI?

There are parallels between human fetish formation and AI alignment challenges, suggesting that understanding how misinterpretations of evolutionary cues lead to fetishes might offer insights into AI generalization problems. The article explores the potential for this analogy to inform AI training and interpretability, acknowledging the differences between biological evolution and AI programming.

40% OF WORKERS WILL HAVE TO RESKILL, ACCORDING TO IBM

According to a new survey from IBM, 40% of the workforce will have to reskill as a result of implementing AI and automation over the next three years.

FLEX FLOW LLM DEPLOYMENT FRAMEWORK

Powerful software to enable deployment of language models in production systems.

3 TRILLION OPEN TOKEN DATASET RELEASED

There are many open datasets, but few of sufficient scale to train a frontier model. The Allen Institute for AI’s Dolma dataset aims to enable researchers to study data effects at scale.

OPENAI’S CONTENT MODERATION

OpenAI proposes using GPT-4 for content policy development and content moderation decisions across digital platforms, taking the burden away from humans and allowing more consistent labeling of content and faster feedback loops. This begs the question, should AI be the authority on who can say what?

SIGMOID LOSS FOR LANGUAGE IMAGE PRETRAINING

CLIP was somewhat revolutionary for computer vision. It has some extremely clever algorithmic tricks (e.g., large in-batch negatives) that make it perform well. This paper questions many of those tricks and finds a simpler training setup that outperforms CLIP for the same task.

AN EFFICIENT WAY TO EVALUATE TEXT-TO-IMAGE GENERATIONS

Current methods for judging the quality of images created from text have some problems, like not truly capturing how good the image looks or how well it matches the text. This paper introduces a new method that estimates how likely a created image matches a given text, focusing on the most important parts of the image. It does this efficiently with fewer samples.

IMPROVING 3D DETECTION IN NEW ENVIRONMENTS

Detecting objects in 3D using LiDAR has challenges when moving to new places because of differences in how data looks. This new method, GPA-3D, uses special reference points to understand 3D shapes better and helps the detector to work well even in unfamiliar areas.

RL ENVIRONMENT FOR ROBOT PIANIST

The robot pianist was an amazing project whereby an agent learned to control a robot hand that played classical piano songs. It is now open sourced for you to train our own.