TLDR AI 2023-03-17

STRIPE AND OPENAI

PWC USING GENERATIVE AI FOR LEGAL

S4 TAKES ON RL

S4 models are a great new long range architecture. This work modifies them so they can initialize and update their internal state in parallel and then explores the modifications on a meta learning task. They found some promising results that point to S4 and other state space models potentially being used as the default RL architecture over LSTMS.

INVESTIGATING GANSFORMER: A VARIATION OF GANS FOR IMAGE GENERATION

The article discusses the authors' reproduction and evaluation of the GANformer, a variation of the Generative Adversarial Networks (GANs) used for image generation. They successfully recreated both variations of the model and found differences between their results and those of the authors, as well as two undisclosed variations of the presented procedures.

RE-REND: REAL-TIME RENDERING OF NERFS ACROSS DEVICES

This paper proposes Re-ReND, a method designed to achieve real-time performance of Neural Radiance Field (NeRF) rendering on resource-constrained devices by converting the NeRF into a representation that can be efficiently processed by standard graphics pipelines, distilling the NeRF by extracting the learned density into a mesh, and factorizing the learned color information into a set of matrices that represent the scene’s light field.

NANOT5

The repository contains code for reproducing a pre-trained T5 language model with limited resources in PyTorch. The model is pre-trained on the C4 dataset and fine-tuned on Super-Natural Instructions (SNI). Despite using only one GPU and limited resources, the model achieves ~40 RougeL on the SNI test set, compared to ~42 RougeL of the original HuggingFace model. The contribution is a user-friendly template for NLP research.

LLAMA SUPPORT IN HUGGINGFACE

BIFORMER: VISION TRANSFORMER WITH BI-LEVEL ROUTING ATTENTION

The article discusses the use of attention as a powerful tool for capturing long-range dependencies in vision transformers but highlights the challenge of its high computational burden and memory footprint. To address this issue, the authors propose a dynamic sparse attention via bi-level routing that allows for flexible allocation of computations with content awareness. The proposed BiFormer model, which utilizes the bi-level routing attention, shows good performance and high computational efficiency across various computer vision tasks.

MOZILLA RESPONSIBLE AI CHALLENGE

CHALLENGE: READ ANCIENT SCROLLS WITH ML

The Vesuvius Challenge aims to read an unopened Herculaneum scroll for the first time and is offering $250,000 in prizes, including a $150,000 Grand Prize and $100,000 in Progress Prizes. Contestants receive 3D X-ray scans, papyrus fragments, tutorials, and tools to virtually unwrap the scrolls. They also have access to a Discord server to connect with other contestants and the Vesuvius Challenge team. The competition is open, and the organizers believe that it will help achieve the goal by 2023.

INFERENCE AT THE EDGE

ALPACA.CCP

Alpaca allows you to run a fast ChatGPT-like model locally on your device.

EVALS

Evals is a framework for evaluating OpenAI models and an open-source registry of benchmarks.