Search the archive

LARS

LARS is an application that enables you to run LLMs locally on your device. Upload your own documents and engage in conversations where the LLM grounds its responses with your uploaded content.

HALLUCINATIONS IN DIFFUSION-BASED IMAGE MODELS

This paper investigates why diffusion-based image generation models create "hallucinations" — images that never appeared in the training data.

DEEP DIVE: BEYOND THE BASICS OF RAG (48 MINUTE VIDEO)

The creator of RAGatouille gave an excellent talk discussing how to dramatically improve RAG performance, some of the open challenges, and COLBERT.

LOGICAL REASONING IN LANGUAGE MODELS

Chain of Preference Optimization (CPO) is a method that improves the logical reasoning abilities of large language models (LLMs). By fine-tuning LLMs using search trees from the Tree-of-Thought (ToT) method, CPO aligns the reasoning steps of Chain-of-Thought (CoT) decoding with ToT's optimal paths.

GENERAL INTELLIGENCE

What would it take to make a generally intelligent agent and what are we missing? This post explores the 3 ideas needed to make an agent and posits that we are only a few years away. The author is a researcher at OpenAI.

TOKENCOST

TokenCost helps calculate the USD cost of using major LLM APIs by calculating the estimated cost of prompts and completions.

MINDMATRIX (PRODUCT)

MindMatrix is an AI-powered mind map generator that simplifies the creation and visualization of complex concepts.

GAIANET-NODE

Install and run your own AI agent service.

NVIDIA LAUNCHES GPT-4 QUALITY MODEL

NVIDIA's Nemotron-4 340B is a family of open models that developers can use to generate synthetic data for training LLMs for commercial applications. The state-of-the-art reward model matches the original GPT-4 model and can run on 8 H100s.

APPLE TO ‘PAY' OPENAI FOR CHATGPT THROUGH DISTRIBUTION, NOT CASH

Apple and OpenAI's partnership isn't expected to generate meaningful revenue for either party, at least at the beginning. Apple believes that pushing OpenAI's brand and technology to its devices is of equal or greater value than monetary payments so it isn't paying OpenAI as part of the deal. The deal isn't exclusive - Apple is already discussing offering other chatbots as additional options. Apple eventually plans to make money from AI by striking revenue-sharing agreements with AI partners.

AI WILL MAKE MONEY SOONER THAN YOU'D THINK

This article contains a transcript of an interview with Aidan Gomez, CEO of Cohere, where he discusses Cohere's path to profitability, the benefit of competition in the enterprise space, AI's current abilities, the future of large language models, and more.

MASKED DIFFUSION LANGUAGE MODELS

Simple diffusion model for language modeling. It generates out of order and works quite well.

JAILBREAK ATTACKS ON LARGE LANGUAGE MODELS

Researchers are addressing the vulnerability of Large Language Models to jailbreak attacks, which manipulate models to produce harmful outputs.

EFFICIENT TRANSFORMER WITH KEY-VALUE SHARING

Multi-Layer Key-Value (MLKV) sharing is a new method that significantly reduces memory usage during auto-regressive inference in transformers.

LLAMA 3 8B MATCHES GPT 4 MATH PERFORMANCE WITH MCTS

The current Holy Grail of research is figuring out how to integrate search into language model generation. This work is extremely promising and shows that small models with search can match much more powerful models on math performance.

VIDEO UNDERSTANDING WITH VIDEOGPT+

VideoGPT+ is a model that combines image and video encoders to enhance video understanding. Image encoders capture detailed spatial information, while video encoders provide temporal context.

SCENE GRAPH GENERATION IN SATELLITE IMAGERY

This project introduces a new dataset and framework to improve Scene Graph Generation (SGG) for very-high-resolution satellite imagery (VHR SAI).

FAKE BEAUTY QUEENS CHARM JUDGES AT THE MISS AI PAGEANT

Aiyana Rainbow, a Romanian AI model, is a finalist in the inaugural Miss AI pageant, an event that features AI-generated models on social media. Organized by FanVue, the event is part of "The FanVue World AI Creator Awards," which showcases AI creators' ability to generate engaging content without being the physical face themselves. The winner of Miss AI is set to receive $5,000, public relations support, and mentorship. The results will be announced at the end of June.

NVIDIA'S NEW SALES BOOSTER: THE GLOBAL PUSH FOR NATIONAL AI CHAMPIONS

Governments around the world are boosting their budgets to encourage companies and multinationals to build new data centers and refit old ones to develop AI locally and train large language models in their native languages based on their own citizens' data. Nvidia says that these sovereign AI efforts are expected to bring in almost $10 billion this year. Many governments are concerned about how generative AI will impact their economies. They want sovereign clouds for their AI infrastructure and sensitive data, and US tech companies are eager to build those for them.

ELON MUSK RECONSIDERS PHONE PROJECT AFTER APPLE INTELLIGENCE OPENAI INTEGRATION

Elon Musk responded to Apple's integration of OpenAI ChatGPT on some of its devices by threatening to ban all Apple devices from the premises of his companies.

MICROSOFT AI CEO PEERS INTO OPENAI'S CODE, HIGHLIGHTING AN UNUSUAL RIVALRY

OpenAI, initially created to counter DeepMind's AI advancements, is now closely partnered with Microsoft, which has rights to OpenAI's technology due to significant investments.

LLM.MOJO

A port of Andrej Karpathy's llm.c to Mojo.

AMAZON SAYS IT'LL SPEND $230M ON GENERATIVE AI STARTUPS

Amazon says that it will commit up to $230 million to startups building generative AI-powered applications.

MICROSOFT IS BUILDING AN AI EMPIRE

Microsoft CEO Satya Nadella has diversified Microsoft's AI investments by partnering with AI startups and building an in-house AI team led by Mustafa Suleyman. This strategy includes significant investments, including $1.5 billion in an Abu Dhabi-based AI firm. It aims to create internal AI capabilities potentially rivaling OpenAI.

OPENAI ADDS FORMER NSA CHIEF TO ITS BOARD

OpenAI is adding former NSA head and retired General Paul Nakasone to its board of directors as well as its newly formed Safety and Security Committee.

APPLE MADE ONCE-UNLIKELY DEAL WITH SAM ALTMAN TO CATCH UP IN AI

Apple and OpenAI have forged a partnership to integrate OpenAI's ChatGPT into iOS. The agreement was a key focus for Apple's Worldwide Developers Conference. The partnership is likely a short- to medium-term relationship for Apple, which is developing its own AI features. Apple is still in talks with Google about using Gemini in its devices. The company is also looking beyond chatbots - it is currently working on a mobile robot that can follow users around and handle chores on their behalf and AI features for its AirPods and cameras.

LLM-SQUARED

Sakana AI has employed an evolutionary strategy to find a preference optimization scheme that outperforms DPO. It used a language model to suggest code and then trained models based on that code. After 100 or so generations, it had several proposed variants with extremely high performance.

RECAPTIONING BILLIONS OF IMAGES WITH LLAMA 3

Recaptioning web scraped images with a vision model dramatically improves downstream model performance. This is especially true in models such as CLIP.

CREATING IMMERSIVE AUDIO

This paper introduces DiffRIR, a new framework designed to reconstruct the spatial acoustic characteristics of environments using a small number of room impulse response (RIR) recordings and a planar scene reconstruction.

Use code TLDR60 for a 60% discount →

SAMBA: A SIMPLE MAMBA

Microsoft has trained a Mamba-based model on the Phi 3 dataset. It dramatically outperforms the transformer variant on MMLU, HumanEval, and retrieval.

DIFFUSIONKIT MLX

Framework and tooling for running diffusion models on Apple's MLX framework.

MASKED DIFFUSION MODELS FOR LANGUAGE MODELING

Researchers have demonstrated that masked discrete diffusion models can rival autoregressive methods in language modeling by applying an effective training recipe and introducing a simplified Rao-Blackwellized objective.

GEMINI 1.5 PRO AND 1.5 FLASH GA, 1.5 FLASH TUNING SUPPORT, HIGHER RATE LIMITS, AND MORE API UPDATES

Google AI has announced updates to the Gemini API and Google AI Studio, including the Gemini 1.5 stable release, higher API rate limits, model tuning support, and new JSON schema and mobile support features. The updates enhance developers' options for building at scale with improved customization and efficiency.

AI-GENERATED SOUND EFFECTS ARE HERE

ElevenLabs has introduced a new AI Audio model capable of creating diverse sound effects, tracks, and voices from text prompts. Leveraging Shutterstock's audio library, this collaboration enhances content creation for media professionals by enabling fast, scalable production of high-quality audio. Users can easily generate sounds through ElevenLabs' platform, simplifying the audio design process.

OPENAI DOUBLES ANNUALIZED REVENUE TO $3.4 BILLION

The vast majority of OpenAI's revenue comes from OpenAI's products and services. The company has been working to boost revenue by selling services to enterprise customers.

SPLASH ATTENTION (GITHUB FILE)

Secret new DeepMind kernel in Jax for Sparse Flash Attention.

HUGGING FACE ACQUIRES AGRILLA

Agrilla, a preference fine-tuning and data company, is joining Hugging Face via acquisition.

OPENAI WELCOMES SARAH FRIAR (CFO) AND KEVIN WEIL (CPO)

OpenAI has appointed Sarah Friar as CFO and Kevin Weil as CPO, strengthening its leadership to support the organization's mission in AI research and product development tailored for consumers, developers, and businesses.

STABLE DIFFUSION 3 OPEN WEIGHTS

Stability AI has released SD 3 weights for non-commercial use, with commercial use available on request.

SHUTTERSTOCK IMAGEAI, POWERED BY DATABRICKS

Databricks has partnered with Shutterstock to build a Shutterstock-branded AI image generation model.

OPENAI ANNUALIZED REVENUE DOUBLES

OpenAI has more than doubled its annualized revenue to hit $3.4B.

BERTS ARE IN-CONTEXT GENERATIVE LEARNERS

In an alternative world, BERT models would have been discovered to be in-context learners instead of their decoder-only GPT counterparts. This paper explores when that is true and finds that, unsurprisingly, BERTs are excellent at information retrieval but struggle with knowledge acquisition - likely due to the bidirectional attention mechanism.

TEXTGRAD: AUTOMATIC DIFFERENTIATION WITH TEXT

This paper explores the idea of treating a language model that can update text as a backpropagation system. The researchers find substantial improvements in benchmark performance, not compute matched, against baseline models.

IMPROVE MATHEMATICAL REASONING WITH AUTOMATIC PROCESS SUPERVISION

DeepMind discovered an excellent extension to the human labor-intensive process supervision process. It found that it could automate much of the process with strong base models, leading to substantial mathematical reasoning performance on Gemini Pro tuned models.

AUTOREGRESSIVE LANGUAGE MODELS FOR IMAGE SYNTHESIS

DEEPMIND TORAX

Google's DeepMind has open-sourced its differentiable fusion tokamak simulator written in Python-Jax. The simulator has strong auto-diff capabilities and covers a number of extremely powerful PDEs.