TLDR Index
TLDR AI

TLDR AI 2023-08-21

6 sections13 entriesinline urlcomplete parse

Parse notes

  • Subscription/account footer truncated before parsefooter_truncated

01GENERAL

Sponsored

STARTUPS

If you're an investor, you should be asking how AI is going to impact your portfolio. If you aren't sure, we have a solution: Fundrise just launched a first-of-its-kind venture capital product

fundrise.comSponsor

02HEADLINES & LAUNCHES

3 TRILLION OPEN TOKEN DATASET RELEASED

There are many open datasets, but few of sufficient scale to train a frontier model. The Allen Institute for AI’s Dolma dataset aims to enable researchers to study data effects at scale.

blog.allenai.org4 min read

OPENAI’S CONTENT MODERATION

OpenAI proposes using GPT-4 for content policy development and content moderation decisions across digital platforms, taking the burden away from humans and allowing more consistent labeling of content and faster feedback loops. This begs the question, should AI be the authority on who can say what?

openai.com5 min read

03RESEARCH & INNOVATION

SIGMOID LOSS FOR LANGUAGE IMAGE PRETRAINING

CLIP was somewhat revolutionary for computer vision. It has some extremely clever algorithmic tricks (e.g., large in-batch negatives) that make it perform well. This paper questions many of those tricks and finds a simpler training setup that outperforms CLIP for the same task.

arxiv.org30 min read

AN EFFICIENT WAY TO EVALUATE TEXT-TO-IMAGE GENERATIONS

Current methods for judging the quality of images created from text have some problems, like not truly capturing how good the image looks or how well it matches the text. This paper introduces a new method that estimates how likely a created image matches a given text, focusing on the most important parts of the image. It does this efficiently with fewer samples.

arxiv.org17 min read

IMPROVING 3D DETECTION IN NEW ENVIRONMENTS

Detecting objects in 3D using LiDAR has challenges when moving to new places because of differences in how data looks. This new method, GPA-3D, uses special reference points to understand 3D shapes better and helps the detector to work well even in unfamiliar areas.

arxiv.org16 min read

04ENGINEERING & RESOURCES

RL ENVIRONMENT FOR ROBOT PIANIST

The robot pianist was an amazing project whereby an agent learned to control a robot hand that played classical piano songs. It is now open sourced for you to train our own.

github.comGitHub repository

CODE FOR BAYESIAN FLOW NETWORKS

Bayesian Flow Networks are an exciting new architecture and training algorithm. This is a simple, unofficial, reproduction of the paper. The goal is to build a codebase that can scale to GPT-2 sized models.

github.comGitHub repository

TXTAI

txtai is an all-in-one embeddings database for semantic search, LLM orchestration, and language model workflows.

github.comGitHub repository

05MISCELLANEOUS

ANTI-HYPE LLM READING LIST

It's hard to know what is real in the current AI kerfuffle. This list contains foundational papers, interesting open questions, and a guide for gaining deeper insights into the space.

gist.github.com20 min read

YOU PROBABLY DON’T NEED TO FINE TUNE AN LLM

Fine-tuning is often not necessary for most LLM applications. It may be better to use few-shot prompting or retrieval-augmented generation (RAG) instead. Few-shot prompting involves providing the LLM with examples of the desired output, while RAG involves querying a vector database for information that the LLM was not trained on.

tidepool.so8 min read

GOOGLE’S PROJECT IDX

An AI-enabled browser-based development environment for building full-stack web and multiplatform apps.

techcrunch.com2 min read