The Daily Index — Thursday, June 22, 2023 · TLDR Index

Across today’s newsletters

Today’s Index

TLDR AI

PIXAR USED AI TO STOKE ELEMENTAL'S FLAME

Pixar turned to AI to solve a unique animation challenge for its new movie 'Elemental', using AI to animate a character made of fire. Paul Kanyuk, a technical supervisor at Pixar, applied Neural Style Transfer (NST) technology to animate the fiery character, Ember. Despite the hefty computational demand, the team achieved a 20X speed-up by leveraging GPUs, heralding a new intersection of art and technology in animation.

archive.ph · 6 min

TLDR AI

SYNTHETIC DATA AND NEXT LEVEL CURATION GIVE BIG HUMANEVAL BOOST

Textbooks may be all you need for extreme AI performance. By using a highly curated set of data and synthetic textbooks with exercises, a 1.3B parameter model achieves 50+ on HumanEval, a challenging code benchmark. For context, this rivals ChatGPT and far surpasses the original 12B Codex model. This paper follows the Tiny Stories line of work and has lots of nice tidbits.

arxiv.org · 18 min

TLDR AI

IN LANGUAGE MODELS

Language models often hallucinate/confabulate information. This is likely because they are trained to imitate expert sequences which in turn causes them to treat their own (potentially wrong) generations as ground truth. Therefore, as sequence lengths grow, models are more likely to make stuff up. However, if you treat it as a Reinforcement Learning problem and find the next most probable action - while including a backspace action - you can dramatically reduce these hallucination issues.

arxiv.org · 26 min

TLDR AI

ANT IS DEVELOPING AN LLM

Jack Ma-backed Ant Group Co. is developing large-language model technology that will power ChatGPT-style services.

archive.ph · 1 min