TLDR Index

Search the archive

Full-text search across article titles and summaries. Refine with sector, year, content type, and reading-time filters.

Results

16,678 matches

THE CAPACITY FOR MORAL SELF CORRECTION IN LARGE LANGUAGE MODELS

Properly aligned models are more helpful and less harmful than their purely self supervised counterparts. This alignment process leads to more emergent phenomena. This paper from Anthropic explores the ability of RLHF aligned models to self correct based on moral feedback. They find that the simple prompt “Please ensure that your answer is unbiased and does not rely on stereotyping” reduces bias on some benchmarks by as much as 84%!

2023.02.17
arxiv.org30 min readTLDR AI

SHAPING REINFORCEMENT LEARNING EXPLORATION WITH LARGE LANGUAGE MODELS

Exploring with LLMs" (ELLM) is a new method that uses a language model to shape exploration in reinforcement learning algorithms. It rewards agents for achieving goals suggested by a language model prompted with a description of the agent’s current state. ELLM guides agents toward human-meaningful and useful behaviors without requiring human input. It has been evaluated in the Crafter game environment and the Housekeep robotic simulator, showing that ELLM-trained agents perform better on downstream tasks and have better coverage of common-sense behaviors during pre-training.

2023.02.17
arxiv.org15 min readTLDR AI

TOKEN MERGING: YOUR VISION TRANSFORMER, BUT FASTER

Transformers have taken over vision recently with advances such as ViT. One of the main innovations has been representing images as a set of visual tokens. This repository from Meta is a companion to their paper [https://arxiv.org/abs/2210.09461?utm_source=tldrai] about speeding up and decreasing the memory footprint of the tokenization process by slowly merging similar tokens throughout training.

2023.02.17
github.comGitHub repositoryTLDR AI

REVERSE-ENGINEERED API OF MICROSOFT'S BING CHAT

The git repository is a reverse-engineered API of Microsoft's Bing Chat, which allows developers to build their own chatbot clients using the service's natural language processing capabilities.

2023.02.17
github.com3 min readTLDR AI

TRAINING

The article provides an overview of recent progress in acceleration arithmetic and hardware, with a focus on techniques that save computation and memory costs for intermediate tensors during training. It also discusses challenges and areas for future research.

2023.02.17
arxiv.org9 min readTLDR AI

WRITEGPT IS YOUR EVERYDAY WRITING ASSISTANT

writeGPT is designed to overcome unproductive browsing habits by plugging ChatGPT capabilities on any website and search engine while facilitating effective prompt engineering to read, write, rewrite, reply to emails, code, and research any content.

2023.02.16
producthunt.com3 min readTLDR AI

EVOLUTIONARY ALGORITHM DISCOVERS OPTIMIZER THAT OUTPERFORMS ADAM

Adam has been the workhorse for AI optimization since its release. Recent advances in AutoML from Google have shown that simpler mathematical functions might be discovered by computers for use by humans! This work is one such example where a simple optimizer was discovered that has a reduced memory footprint. It improves model performance up to 2% and works well at high batch sizes (e.g., Large Language Model training regime). There’s even code provided.

2023.02.16
arxiv.org45 min readTLDR AI

ON ITS OWN? FINDINGS FROM A COMPREHENSIVE STUDY

The paper presents an empirical analysis of ChatGPT's ability to perform multiple natural language processing tasks without adaptation on downstream data. The authors evaluate ChatGPT on 20 NLP datasets and find that the model performs well on tasks that require reasoning but struggles with tasks like sequence tagging. The article includes in-depth analysis through qualitative case studies.

2023.02.16
arxiv.org12 min readTLDR AI

2023.02.16
dicksonneoh.com4 min readTLDR AI

MCTX BY DEEPMIND

Mctx is an open-source version of the DeepMind programs AlphaZero, MuZero, and Gumbel MuZero.

2023.02.16
github.comGitHub repositoryTLDR AI

AI DRUG DEVELOPMENT

AI is enabling the design of revolutionary new drugs previously thought to be impossible. Now, scientists have to test the effectiveness of these drugs.

2023.02.16
archive.ph12 min readTLDR AI

2023.02.16
fortune.com3 min readTLDR AI