TLDR Index
TLDR AI

TLDR AI 2023-03-06

5 sections16 entriesinline urlcomplete parse

Parse notes

  • Subscription/account footer truncated before parsefooter_truncated

01HEADLINES & LAUNCHES

200-YEAR-OLD MATH OPENS UP AI’S MYSTERIOUS BLACK BOX

Whether it’s designing microchips or dreaming up new proteins, sometimes it seems like neural networks can do anything. Infamously, however, these brain-inspired AI systems work in mysterious ways, raising concerns that what they are doing might not make any sense. New research suggests that 200-year-old math could help shed light on how neural networks perform complex tasks such as predicting climate or modeling turbulence, a new study finds. This in turn could help boost the accuracy of neural networks and the speed at which they learn, researchers say.

spectrum.ieee.org4 min read

02RESEARCH & INNOVATION

MAXIMIZATION

Diffusion is currently one of the best generative methods in terms of sample quality. This comes from the nice properties of the loss and denoising process. In the community, different weighting schemes are used for noise per step which leads to different empirical results. It turns out that these non-uniform weighting schemes can be understood from the lens of likelihood maximization. This is a great step towards a deeper theoretical understanding of this modern workhorse.

arxiv.org22 min read

GUIDE TEXT GENERATION WITH GROUNDING FOR ROBOTIC CONTROL

Applying large language models to robots is challenging due to their lack of experience with the physical world. To overcome this, a guided decoding strategy is used to construct an action sequence that is both likely according to the language model and realizable in the environment. This strategy solves complex tasks in a robotic setting by leveraging the knowledge of both models. They compare their work to SayCan.

grounded-decoding.github.io7 min read

UNLIMITED-SIZE DIFFUSION RESTORATION

This paper discusses the use of diffusion models for zero-shot image restoration and proposes a method to handle images of arbitrary sizes while maintaining the excellent characteristics of zero-shot. The proposed method, called Mask-Shift Restoration, addresses local incoherence, and Hierarchical Restoration alleviates out-of-domain issues. These simple, parameter-free approaches can be used not only for image restoration but also for image generation of unlimited sizes.

arxiv.org12 min read

03ENGINEERING & RESOURCES

EARLY DROPOUT FOR MITIGATING UNDERFITTING IN NEURAL NETWORKS

The study shows that early dropout, applied only during the initial phases of training, can mitigate underfitting by reducing the directional variance of gradients and aligning them with the entire dataset’s gradient. The proposed method consistently improves generalization accuracy in various vision tasks and encourages further research on regularization in deep learning.

github.comGitHub repository

CONTEXT CLUSTERS: A NEW PARADIGM FOR VISUAL REPRESENTATION

This work introduces Context Clusters (CoCs), a new paradigm for visual representation that views an image as a set of unorganized points and extracts features via a simplified clustering algorithm. CoCs are convolution- and attention-free and only rely on clustering for spatial interaction. Despite not targeting state-of-the-art performance, CoCs achieve comparable or even better results than ConvNets or ViTs on several benchmarks.

github.comGitHub repository

04MISCELLANEOUS

THE INSIDE STORY OF HOW CHATGPT WAS BUILT

To get the inside story behind the ChatGPT — how it was made, how OpenAI has been updating it since release, and how its makers feel about its success — the author talked to four people who helped build it. They are Sandhini Agarwal (works on policy at OpenAI), Liam Fedus (scientist who worked on ChatGPT), John Schulman (cofounder of OpenAI), and Jan Leike (leader of OpenAI’s alignment team)

archive.ph10 min read