TLDR AI 2023-02-28

VOICEMOD RAISES $14.5M TO RIDE THE GENERATIVE AI (SONIC)BOOM

Voicemod has become the leading creator in real-time voice changing and soundboard technology. Their mission is to enable everyone to express themselves through sound. They’ve built expressive and immersive audio tools, making it easy to create unique sonic identities and enable interactions with personalized sounds. This will be huge as the podcast, voiceover, and audiobook spaces have blown up the last few years.

DEEP GRAPH LIBRARY REACHES 1.0

GENERATE CODE BY RETRIEVING DOCS

With code interfaces regularly changing and the limitation of in-context learning, there is a strong need to be able to update program synthesis performance without expensive data collection and model retraining. This work suggests that using documentation can improve the generated code of CodeT5. The results are good with ~3% improvement Pass@1. This isn’t a foundational change, but may be a useful trick for practitioners.

LANGUAGE IS NOT ALL YOU NEED: ALIGNING PERCEPTION WITH LANGUAGE MODELS

KOSMOS-12 is a Multimodal Large Language Model that can learn in context, follow instructions, and perceive general modalities. It achieves impressive performance on a range of tasks, including language understanding, perception-language, and vision tasks. The model was trained on web-scale multimodal corpora and benefits from cross-modal transfer. The authors also introduce a dataset for diagnosing the nonverbal reasoning capability of MLLMs.

ATTENTION GUIDANCE

Text-guided diffusion models struggle to compose scenes with multiple objects in specific positions, which is crucial in storytelling. To address this, the authors propose Directed Diffusion, a method that provides positional control over multiple objects by injecting "activation" at desired positions in cross-attention maps while attenuating the rest. This method can be used with an existing pre-trained model and requires only a few lines of implementation.

CHATLLAMA: CHATGPT BASED ON META’S LLAMA MODELS

PREDICTION

The authors propose voxformer, a framework proposed to enable AI systems to imagine the complete 3D geometry of occluded objects and scenes from 2D images. VoxFormer uses a two-stage design where a sparse set of visible and occupied voxel queries from depth estimation is followed by a densification stage that generates dense 3D voxels. The framework adopts a masked autoencoder design to propagate the information to all the voxels by self-attention. Experiments on SemanticKITTI show that VoxFormer outperforms the state of the art with a relative improvement of 20.0% in geometry and 18.1% in semantics and reduces GPU memory during training by ~45% to less than 16GB.

40 YEARS OF AI COMPUTE

A nice compilation of various trends across the years in AI. Required computation doubles every 9 months while the number of parameters doubles every 18 months. Hardware improvements may improve through 2031. It doesn’t make sense to train models longer than 15 months. This is not exclusive to language but also includes vision and RL. Interactive charts included.

CALM DOWN, THERE IS NO CONSCIOUS AI

META FORMING AN AI PRODUCT TEAM