TLDR AI 2023-06-22

INSIDE THE WHITE HOUSE’S URGENT PUSH TO REGULATE AI

PIXAR USED AI TO STOKE ELEMENTAL'S FLAME

Pixar turned to AI to solve a unique animation challenge for its new movie 'Elemental', using AI to animate a character made of fire. Paul Kanyuk, a technical supervisor at Pixar, applied Neural Style Transfer (NST) technology to animate the fiery character, Ember. Despite the hefty computational demand, the team achieved a 20X speed-up by leveraging GPUs, heralding a new intersection of art and technology in animation.

CHINA'S UNDERGROUND MARKET FOR NVIDIA AI CHIPS

SYNTHETIC DATA AND NEXT LEVEL CURATION GIVE BIG HUMANEVAL BOOST

Textbooks may be all you need for extreme AI performance. By using a highly curated set of data and synthetic textbooks with exercises, a 1.3B parameter model achieves 50+ on HumanEval, a challenging code benchmark. For context, this rivals ChatGPT and far surpasses the original 12B Codex model. This paper follows the Tiny Stories line of work and has lots of nice tidbits.

IN LANGUAGE MODELS

Language models often hallucinate/confabulate information. This is likely because they are trained to imitate expert sequences which in turn causes them to treat their own (potentially wrong) generations as ground truth. Therefore, as sequence lengths grow, models are more likely to make stuff up. However, if you treat it as a Reinforcement Learning problem and find the next most probable action - while including a backspace action - you can dramatically reduce these hallucination issues.

HIGH THROUGHPUT AND MEMORY EFFICIENT TRANSFORMERS

As Transformers have started to gain more and more traction, they are being served far more often than trained. This can be quite slow and expensive, but with some clever optimizations around memory layout on GPU, they get many multiples faster.

WANDA: A NEW PRUNING METHOD FOR LARGE LANGUAGE MODELS

The repo presents Wanda, a unique method for reducing the complexity of large language models by selectively dropping less significant network weights. Unlike other methods, Wanda doesn't require retraining or intensive computation, providing effective model pruning while maintaining performance.

STANDARDIZING SPATIO-TEMPORAL PREDICTIVE LEARNING

This repo introduces OpenSTL, a benchmark for models that predict future patterns based on past frames. It categorizes approaches and evaluates them on various datasets.

AI IS A LOT OF WORK

ANT IS DEVELOPING AN LLM

Jack Ma-backed Ant Group Co. is developing large-language model technology that will power ChatGPT-style services.

FRESHWORKS UNVEILS NEW GENERATIVE AI ENHANCEMENTS

SECRET INVASION’S OPENING CREDITS ARE AI-MADE

SCHUMER UNVEILS STRATEGY TO REGULATE AI