Search the archive

US REGULATORS WARN THEY ALREADY HAVE THE POWER TO GO AFTER AI BIAS

MINIGPT-4

MiniGPT-4 enhances vision-language understanding using advanced large language models.

AI IS REALLY GOOD FOR LOW SKILLED WORKERS

When a conversational AI tool is used to support customer support they get a big boost, somewhere around 14% on average with an increase in employee AND customer satisfaction. Interestingly, it seems like the biggest bump is for lower skilled support agents. This might be because they now have access to a broad knowledge base that was hard to get before. This also increased employee retention. They don’t touch on wage effects of these tools, but merely explore potential productivity gains.

GEN-1 ON IOS

GOOGLE RELEASES SECURITY LLM

WHAT HAPPENS IF WE GIVE LANGUAGE MODELS ANXIETY?

There are certain types of phrases that are used in computational psychiatry and are known to elicit or measure anxiety. As it turns out, if we use these and other emotion causing prompts, ChatGPT responds strongly. It also increases its ability to explore and its bias in some settings.

IS ADAM ACTUALLY THE BEST, OR DID WE MAKE IT THE BEST?

SIMPLE PARAMETER-EFFICIENT FINE-TUNING

DiffFit, a parameter-efficient strategy for fine-tuning large pre-trained diffusion models, enables fast adaptation to new domains while significantly reducing training time and storage costs. Achieving superior performance on downstream datasets, DiffFit sets a new state-of-the-art FID on the ImageNet 512x512 benchmark and demonstrates its potential for adapting low-resolution models to high-resolution ones with minimal cost.

CHEETAH

Cheetah is an AI-powered macOS app designed to assist users during remote software engineering interviews by providing real-time, discreet coaching and live coding platform integration.

VisFusion is a visibility-aware online 3D scene reconstruction method from posed monocular videos, improving feature fusion by inferring visibility using a similarity matrix calculated from projected features in each image pair. The approach employs a coarse-to-fine pipeline with a volume sparsification process, preserving fine details by sparsifying voxels locally along each visual ray, and enhancing TSDF predictions through learning residuals across scales.

SILVR: GUIDED DIFFUSION FOR MOLECULE GENERATION

The selective iterative latent variable refinement (SILVR) method conditions diffusion-based equivariant generative models without retraining, generating novel molecules that fit a protein's binding site based on fragment hits. Using the SARS-CoV-2 Main protease fragments as a reference dataset, SILVR demonstrates the ability to generate new molecules with similar shapes to original fragments, fitting the binding site without knowledge of the protein, and merging up to three fragments into a new molecule without affecting the underlying generative model's quality.

HUMANS

WHY AGENTS ARE THE NEXT KILLER AI APP

This article makes the case that autonomous agents like Auto-GPT are the next big use-case for AI. Auto-GPT (and its younger sibling BabyAGI) are independently developed Python projects that have caught enormous popularity, with Auto-GPT trending #1 on Twitter and GitHub in the past 2 weeks.

THE AI DOOMERS’ PLAYBOOK

GRIMES IS OK WITH AI GENERATED MUSIC

THE UK IS SPENDING £100 MILLION TO DEVELOP ITS OWN SOVEREIGN AI

WEB LLM

Web LLM brings language model chats directly onto web browsers.

BARD CAN HELP YOU CODE

Google’s language model Bard, when launched, refused to answer coding questions. That has changed now. It can help you code in a huge variety of languages and does a pretty good job! There hasn’t been any official validation released, so we don’t know how it compares to other program synthesis models, but it’s always nice to have options.

MIND AND BRAIN

INTRODUCING W&B PROMPTS

THEORY OF ADAM INSTABILITY

The Meta folks have been training some big models recently. Part of the pain of that is loss spikes. This is where training diverges and usually requires manual intervention to get things back on track. This paper outlines a hypothesis for these spikes. They claim that Adam is the culprit. Maybe some of the new optimizers like Lion or Shampoo can help?

REAL-TIME VIEW SYNTHESIS

This paper presents a novel method that distills and bakes Neural Radiance Fields (NeRFs) into efficient mesh-based neural representations, enabling real-time rendering and compatibility with parallel graphics pipelines. The approach uses a two-layer duplex mesh, screen-space convolutions, and a multiview distillation optimization strategy, resulting in superior performance and effectiveness demonstrated through experiments on standard datasets.

MULTI-MODAL FACE GENERATION AND EDITING

VOICE CLONING

Transformers continue their march into all modalities, this time taking on audio with a cool voice cloning tool. There are some limitations, but with just a few seconds of audio, you can get a pretty great representation of your voice.

LLAMA ACADEMY

It seems like everywhere you look people are building retrieval augmentation into language models. Here is another tool to do that, using the almost-open sourced Llama models. This one has a nice recipe you can follow to teach the model about novel APIs and SDKs.

IS AI-GENERATED MUSIC LEGAL?

PROMPT ENGINEERING VS BLIND PROMPTING

CHATGPT IS VERY EXPENSIVE TO MAINTAIN

ROBOTS AND AI INCREASES PACE OF RESEARCH 100X

AN AI DEVICE THAT PROJECTS PHONE CALLS ONTO YOUR HAND

SPEECHGPT

SpeechGPT is a web application that enables you to converse with ChatGPT.

NEXT LEVEL TEXT-TO-VIDEO SYNTHESIS

TRANSFORMER MATH

How many GPUs do you need? How wide should your model be? These are regularly asked questions with fairly simple calculations. This post walks through these and a bunch more interesting questions involving Transformers and the math needed to train good ones. One interesting take away is that models should not be trained for less than 200B tokens regardless of size.

VISUAL INSTRUCTION TUNING

The paper proposes using machine-generated instruction-following data to instruction tune large language models (LLMs) for multimodal tasks. This resulted in LLaVA, a large multimodal model that combines a vision encoder and language-only GPT-4. LLaVA exhibits impressive multimodal chat abilities and achieved a new state-of-the-art accuracy of 92.53% when fine-tuned on Science QA in synergy with GPT-4.

LOW-CODE LLM: VISUAL PROGRAMMING OVER LLMS

The paper proposes Low-code LLM, a human-LLM interaction framework that uses low-code visual programming interactions to achieve more controllable responses for complex tasks. Low-code LLM includes a Planning LLM and an Executing LLM and offers controllable generation results and user-friendly interaction. The approach is demonstrated through four applications.

PROMPTR

Promptr is a CLI tool for operating on your codebase using GPT. Promptr dynamically includes one or more files into your GPT prompts, and it can optionally parse and apply the changes that GPT suggests to your codebase. Several prompt templates are included for various purposes, and users can create their own templates.

CLAP MODEL WEIGHTS RELEASED

RADIANCE FIELD

Lift3D, an inverted 2D-to-3D generation framework, addresses the limitations of NeRF-based 3D GANs by offering photorealistic output with adaptable resolution and accurate 3D annotations for downstream tasks. The framework is evaluated on autonomous driving datasets, demonstrating its ability to enhance the performance of 3D object detectors.

90% OF MY SKILLS ARE NOW WORTH $0

GPT-4 WILL HUNT FOR TRENDS IN MEDICAL RECORDS

ATLASSIAN RELEASES AI TOOLS

Atlassian Corp. unveiled new artificial intelligence features for its workplace collaboration tools, making it the latest technology company to utilize OpenAI’s generative AI models.

INSIDE THE AI TALENT WARS

Tech companies are now ransacking university AI programs in search of talent.

LLAVA

LLaVA provides visual instruction tuning towards large language and vision models with GPT-4 level capabilities.

ARTIST REJECTS PHOTO PRIZE AFTER AI-GENERATED IMAGE WINS AWARD

SERVICES

LEARNING TO COMPRESS PROMPTS WITH GIST TOKENS

A new technique called "gisting" has been introduced as a way to utilize multitask capabilities of language models without requiring prompts or model retraining for each task. Gisting compresses prompts into smaller sets of tokens that can be reused, resulting in computational efficiency, FLOP reductions, and speedups without significant loss in output quality. This technique was tested on two LMs, achieving up to 26x compression of prompts and up to 40% FLOPs reductions.

DETRS BEAT YOLOS ON REAL-TIME OBJECT DETECTION

A new real-time end-to-end object detector called Real-Time Detection Transformer (RT-DETR) has been proposed to address the high computational cost issue of end-to-end transformer-based detectors (DETRs), which limits their practical application. RT-DETR uses an efficient hybrid encoder, IoU-aware query selection, and supports flexible adjustment of inference speed without retraining. RT-DETR achieves up to 54.8% AP on COCO val2017 and 74 FPS on T4 GPU, outperforming YOLO detectors and DINO-Deformable-DETR-R50 in both speed and accuracy.

STREET FIGHTER AI

There has been recent work in making RL agents less afraid. One paper specifically talks about mitigating cowardice in models trained in RL. This is interesting in the realm of self-driving cars and terrifying in other potential applications. This repository shows how we can now train RL agents to win in a fighting game. The full training and evaluation code is available.