Parse notes
While the content of this piece is a bit sensationalized, the fact that Dr. Hinton has left Google is notable. He claimed, on Twitter, that his goal was to distance himself from Google so he could speak more openly about the potential harms of AI. He claims his previous employer acted responsibly with AI product releases.
Llama is a language model from Meta that was released under a mostly open license. It was trained for much longer than GPT-3 and even longer than was optimally suggested by Deepmind’s Chinchilla. It does ok on medical tasks, but when tuned more scores improve by 30%+. Lots of folks have been talking about Parameter Efficient Fine-Tuning which only updates a fraction of the underlying model. This paper shows that while it is cheaper to use PEFT, performance is sometimes significantly worse.
Diffusion models are a class of machine learning models that learn to slowly remove noise from data to produce high quality output. The Stable Diffusion text-to-image model is an example of such a model. One challenge with these models is setting constraints on the output (imagine an application where you only want to use blue pixels). This paper introduces an inequality constraint on the output so that you can produce diffusion paths for anything within the approved region. They showcase examples of diffusion models for biomedical applications where constraints are extremely tight.
The paper explores the potential of using large-scale text-to-image diffusion models for generative data augmentation to improve challenging discriminative tasks. The study shows that fine-tuned class-conditional models generated by these models have state-of-the-art FID, Inception Score, and Classification Accuracy Scores. Augmenting the ImageNet training set with samples from the generated models results in significant improvements in ImageNet classification accuracy over strong ResNet and Vision Transformer baselines.
The Natural Speech 2 project aims to advance the state of the art in speech recognition by developing new algorithms and models that are better equipped to handle the inherent complexities and nuances of human speech, ultimately improving the performance and usability of speech-based AI systems.
The godot-dodo project presents a pipeline to finetune open source language models on human-created, language-specific code retrieved from GitHub.
Sponsored
An open-source version of GPT.
NewsBytes is an AI generated podcast that covers global headlines in roughly 5 minutes.