Search the archive

ALPACA.CCP

Alpaca allows you to run a fast ChatGPT-like model locally on your device.

EVALS

Evals is a framework for evaluating OpenAI models and an open-source registry of benchmarks.

GPT-4 IS EXCITING AND SCARY

The article discusses the author's experience testing GPT-4 by asking it a variety of questions — to help with a tax problem, to come up with a new word, to plan a birthday party. The author found that GPT-4 was capable of a wider range of tasks, more fluid in its responses, and seemed to have more guardrails in place than its predecessor. GPT-4 also scored high on a variety of tests, including the Uniform Bar Exam and the Biology Olympiad. While the article acknowledges the positive potential of GPT-4, it also discusses the negative side — the thought of AI being smarter than humans, ethical implications of such technology, and the potential for misuse. Also, another correction from yesterday’s newsletter: GPT-4 can take both _IMAGES_ and text as inputs, but not video. Sorry for the brain fart! Maybe I should have had GPT-4 proofread it :)

ADEPT AI RAISES $350M IN SERIES B FUNDING

Adept has raised $350M in new funding as part of their Series B led by General Catalyst and Spark Capital. Adept is a research and product AI lab that automates software processes for developers. It's similar to robotic process automation (RPA), or software robots that use a combination of automation, computer vision, and machine learning to automate repetitive tasks like filing forms and responding to emails. Adept is creating general intelligence by allowing humans and computers to collaborate creatively.

GOVERNMENTS

REINFORCEMENT LEARNING FROM PRIVACY FEEDBACK

IN LARGE LANGUAGE MODELS

This paper proposes a solution to the challenge of fine-tuning and task-specific prompt engineering that can limit the generalization abilities of Large Language Models (LLMs). The proposed method, UPRISE, uses a lightweight and versatile retriever to automatically retrieve prompts for a given zero-shot task input, and demonstrates universality in a cross-task and cross-model scenario. The method is tested on a diverse set of tasks, using a small frozen LLM for tuning and testing on much larger LLMs, and is shown to mitigate the hallucination problem in experiments with ChatGPT, suggesting potential to improve even the strongest LLMs.

DEEPMIM: DEEP SUPERVISION FOR MASKED IMAGE MODELING

This paper introduces DeepMIM, a deep supervision technique that significantly boosts the representation capability of each layer in Vision Transformers for masked image modeling. The results show that DeepMIM accelerates model convergence, expands attention diversities, and achieves state-of-the-art performance on various downstream tasks.

PIP INSTALL PYTORCH 2.0

Announced in December, this new version of pytorch is now stable and available for easy installation. With a slew of new features, speed improvements, and interfaces - it’s an exciting time to be using the most popular deep learning framework out there.

EDIT-A-VIDEO: A TEXT-TO-VIDEO APPROACH FOR VIDEO EDITING

This paper presents Edit-A-Video, a framework for video editing based on text-to-video (TTV) models adapted from diffusion-based text-to-image (TTI) models. The proposed framework inflates the 2D model into a 3D model, and inverts the source video into the noise to edit with a target text prompt and attention map injection, while also mitigating the background inconsistency problem through a novel mask blending method. Experimental results show the superiority of the proposed method in terms of background consistency, text alignment, and video editing quality compared to baselines.

GOOGLE PARTNERS WITH GRETEL TO ADD SYNTHETIC DATA INTO VERTEX AI

HOW SIRI, ALEXA, AND GOOGLE ASSISTANT LOST THE AI RACE

Virtual assistants like Siri, Alexa, and Google Assistant had more than a decade head start to become indispensable. But they were hampered by clunky design and miscalculations, leaving room for chatbots like ChatGPT to rise.

HERE’S WHAT IT WOULD TAKE TO SLOW OR STOP AI

LINKEDIN IS ADDING AI TOOLS

LLAMA-RS

LLaMA-rs is a Rust port of LLaMA.

GPT-4-FOR-CODE

This GitHub repo contains a few impressive examples of GPT-4 coding.

OPENAI RELEASES GPT-4

It’s finally here! GPT-4 is a smarter, safer, and more robust version of its predecessor. GPT-4 is safer and more aligned, but still has limitations. It is 82% less likely to respond to requests for disallowed content and 40% more likely to produce factual responses than GPT-3.5 based on internal evaluations. Companies like Duolingo, Khan Academy, and Stripe are already using the service. Also, a correction from yesterday’s newsletter: previous rumors that GPT-4 will include text-to-video capabilities are actually not true. GPT-4 has not yet made the leap to multimodal outputs like video, but it can take in both video and text as inputs!

GOOGLE UNVEILS NEW PALM API AND MAKERSUITE TOOL

ANTHROPIC LAUNCHES CLAUDE

NEW PRE-TRAINING PARADIGM IMPROVES LANGUAGE MODELS' CAPABILITIES

This paper proposes a new pre-training paradigm that improves the training data efficiency and capabilities of language models in the infilling task. The paradigm includes a bidirectional training objective that aligns the predictions of left-to-right and right-to-left LMs and a bidirectional inference procedure. The approach is evaluated on programming and natural language models and outperforms strong baselines.

MODEL-TUNING VIA PROMPTS: IMPROVING ADVERSARIAL ROBUSTNESS IN NLP

The MVP method demonstrates surprising gains in adversarial robustness, improving performance against adversarial word-level synonym substitutions by an average of 8% over standard methods, and even outperforming state-of-art defenses by 3.5%. By modifying the input with a prompt template instead of modifying the model by appending an MLP head, MVP achieves better results in downstream tasks while maintaining clean accuracy.

DISTRIBUTION FITTING

UniDiffuser is a framework for fitting all distributions relevant to multi-modal data in one model. It achieves this by unifying diffusion models for marginal, conditional, and joint distributions. UniDiffuser is able to perform image, text, text-to-image, image-to-text, and image-text pair generation with high-quality results and competitive quantitative scores.

UNIDIFFUSER

UniDiffuser is a unified diffusion framework to fit all distributions relevant to a set of multi-modal data in one model.

WRITEOUT.AI

Writeout.ai translates audio files using OpenAI’s Whisper API.

MICROSOFT HAS LAID OFF ONE OF ITS RESPONSIBLE AI TEAMS

GOOGLE ANNOUNCES NEW AI FEATURES FOR WORKSPACE

ADVISORS

THE AGE OF INFINITE MISINFORMATION HAS ARRIVED

In this article, Gary Marcus makes the case that bad actors could seize on large language models to engineer falsehoods at unprecedented scale.

ALPACA: A STRONG OPEN-SOURCE INSTRUCTION-FOLLOWING MODEL

Instruction-following models such as GPT-3.5 (text-davinci-003), ChatGPT, Claude, and Bing Chat have become increasingly powerful. However, instruction-following models still have many deficiencies: they can generate false information, propagate social stereotypes, and produce toxic language. To make maximum progress on addressing these pressing problems, it is important for the academic community to engage. Alpaca is a model fine-tuned from the LLaMA 7B model on 52K instruction-following demonstrations.

TOGETHER RELEASES OPENCHATKIT

OpenChatKit provides a powerful, open-source base to create both specialized and general purpose chatbots for various applications. OpenChatKit is Together’s version of OpenAI’s ChatGPT. All of the model’s weights, code, and training data are available online. And it was even trained on 100% carbon-negative compute.

JENSEN'S INEQUALITY

An extremely useful inequality for applied mathematicians and machine learning practitioners. This post explores this often backwards, but useful, inequality. If you're hoping to learn more of the math behind ML, this is a good piece to read.

UNALIGNED FACE MANIPULATION

This paper proposes a solution to the limitations of StyleGAN, which is restricted to cropped aligned faces at fixed image resolution. The solution involves using dilated convolutions to rescale the receptive fields of shallow layers in StyleGAN without altering model parameters, resulting in features that can accommodate variable resolutions and better characterize unaligned faces. Additionally, the paper introduces an encoder that provides the first-layer feature of the extended StyleGAN and validates the effectiveness of the method in various face manipulation tasks, including super-resolution, facial attribute editing, sketch/mask-to-face translation, and face toonification.

MVIMGNET: A LARGE-SCALE DATASET OF MULTI-VIEW IMAGES

The authors introduce MVImgNet, a large-scale dataset of multi-view images for 3D vision. It contains 6.5 million frames from real-world objects and bridges the gap between 2D and 3D vision. MVPNet, a 3D object point cloud dataset, is also introduced.

HYPER LIGHT SPEED GPT

Based on the NanoGPT repo, this code can train a decoder only transformer model to somewhat reasonable perplexity in just 3 minutes. It is designed to be simple to understand, hackable, with fast convergence for researchers to explore their ideas.

GUARDRAILS

Guardrails is a Python package that lets a user add structure, type, and quality guarantees to the outputs of LLMs.

WHISPERBOARD

Whisperboard is an iOS app for recording and transcribing audio on the go, based on OpenAI’s Whisper model.

THE HOT MESS THEORY OF AI MISALIGNMENT

GPT-4 UPGRADE WILL INCLUDE TEXT-TO-VIDEO

According to Microsoft Germany CTO Andreas Braun, OpenAI’s upcoming GPT-4 upgrade will allow users to turn text into video. Braun added that the more powerful AI will be introduced this week, putting an end to speculation over its release. "We will introduce GPT-4 next week, where we will have multimodal models that will offer completely different possibilities – for example, videos," Braun said.

AI CAN FORECAST THE NEXT FOOD CRISIS

GM PLANS TO LET YOU TALK TO YOUR CAR WITH CHATGPT

CHATGPTNET

ChatGPTNet is a ChatGPT integration library for .NET

OPEN SOURCE CHATGPT REPLICATION TRAINED ON 43 MILLION INSTRUCTIONS

One of the reasons chatGPT is so powerful is because of the instruction tuning process. By feeding the model data formatted as instructions, it becomes much more likely to follow instructions that users give it. Openchatkit is a 20 billion parameter instruction tuned Transformer and other useful tools to make an open source replication of the powerful and popular chatbot.

GPT-4 IS COMING NEXT WEEK SAYS MICROSOFT GERMANY

CHERRY PICKING ROBOT WITH ONLY 30 MIN OF INTERACTION DATA

Data efficiency is one of the challenges of many reinforcement learning algorithms. It usually takes millions of environmental interactions before the agent starts to learn. This is often infeasible with real world robotic systems. This work is one example of a system that is more data efficient.

PIXMIM: A SIMPLE AND EFFECTIVE METHOD FOR MASKED IMAGE MODELING

The paper proposes PixMIM, a simple and effective method for Masked Image Modeling (MIM) that addresses two previously overlooked bottlenecks. PixMIM filters high-frequency components from the reconstruction target and adopts a conservative data transform strategy to alleviate the problem of missing foreground in MIM training. The method can be easily integrated into most existing pixel-based MIM approaches, improving three MIM approaches consistently across various downstream tasks without additional computation. PixMIM can serve as a strong baseline for self-supervised learning and provide insights for future improvements of the MIM framework.

RUN STATE OF THE ART LANGUAGE MODELS ON YOUR LAPTOP

By taking advantage of the new Apple neural computing system, you can run the llama models on m1 and m2 computers. You can even get many tokens per second for the smallest model. This points at an exciting future of personalized AI running on consumer grade hardware.

DEAD SIMPLE WAY TO RUN LANGUAGE MODELS

Powered by the above code, this project packages things up nicely and allows you to install and run the llama models with familiar web tools like npx.

TEXT-GENERATION-WEBUI

COMBINE PEFT WITH RLHF

Parameter efficient fine tuning (PEFT) is a way to update only a small fraction of the weights of a model. Reinforcement learning from human feedback (RLHF) is a similar step to instruction tuning that improves model generation quality and model alignment. By combining the two, you can tune large models up to 20 billion parameters on a relatively small GPU with 24GB of on chip memory. This process would usually require multiple A100s.