TLDR Index
TLDR AI

TLDR AI 2024-02-19

6 sections17 entrieslinks blockcomplete parse

Parse notes

  • Subscription/account footer truncated before parsefooter_truncated

01GENERAL

02HEADLINES & LAUNCHES

OPENAI'S NEXT AI PRODUCT COULD BE AFTER YOUR JOB (AGAIN)

OpenAI has been reportedly developing two types of AI agent software for over a year. The first type can be used to automate complex tasks by taking over a customer's device. The second AI agent class handles web-based tasks and can gather public data. It is unclear when the company plans to release these agents.

androidauthority.com2 min read

GEMINI 1.5 PRO

Google released a new MoE model that matches Gemini 1.0 Ultra in performance but scales up to 1m tokens in context while using less compute due to its smaller size. It is natively multimodal.

blog.google12 min read

03RESEARCH & INNOVATION

LONG IS MORE FOR ALIGNMENT

It is often challenging to know which examples should be used when aligning language models using preference data. This work suggests a surprisingly robust baseline - choose the 1,000 longest examples.

arxiv.org28 min read

EXTREME VIDEO COMPRESSION WITH PRE-TRAINED DIFFUSION MODELS

Diffusion models can be repurposed for their broad ‘knowledge’ of the world as they get better at synthesizing images and videos. This paper found a phenomenal 0.02 bits per pixel compression. The key trick here was to measure perceptual similarity along the way and resend an original video frame as needed.

arxiv.org18 min read

IMPROVING MATH SKILLS IN LLMS

Researchers have created OpenMathInstruct-1, a new dataset for training open-source Large Language Models in math, matching the performance of closed-source models. This breakthrough, featuring 1.8 million problem-solution pairs, opens the door for more accessible and competitive math instruction AI tools.

arxiv.org19 min read

04ENGINEERING & RESOURCES

MINBPE

Andrej Karpathy released a minimal, clean, and extensible implementation of the byte pair encoding used in language model tokenizers.

github.comGitHub repository

GPTSCRIPT

GPTScript is a new scripting language that automates interactions with OpenAI large language models. The project's ultimate goal is to create a fully natural language-based programming experience.

github.comGitHub repository

QWEN 1.8B AND 72B LLMS

These models, which look similar to Llama 2, are trained on 3T tokens and excel at a number of tasks. The Qwen team has released chat versions and quants. Excitingly, the models seem to excel at reasoning, math, and code.

github.comGitHub repository

05MISCELLANEOUS

THE DATA REVOLUTION IN VENTURE CAPITAL

Over 75% of public market trades (AUM of $1T+) are now driven by data-driven algorithms, a revolution initiated by hedge funds in the 90s. Today, that data-driven approach is rapidly infiltrating venture capital, with projections indicating that over 75% of VC deal reviews will involve AI and data analytics by 2025, transforming how investments are sourced, evaluated, and managed.

signatureblock.co10 min read

COMMUNITY, COLLABORATION, CREATIVITY IN THE AGE OF AI

How we talk to each other, collaborate, and undertake creative tasks has meaningfully shifted since we introduced software. We’re starting to see the beginnings of another meaningful shift with AI. How substantial this shift will be is being underestimated. Startups born with AI integration in their products from day one will have a huge advantage over existing companies adding it on top of their existing products.

digitalnative.tech11 min read