UNITOK: A UNIFIED TOKENIZER FOR VISUAL GENERATION AND UNDERSTANDING

Summary from the TLDR AI newsletter

This paper addresses the representational gap between visual generation and understanding by proposing UniTok, a discrete visual tokenizer that simultaneously encodes detailed information for generation and semantic content for understanding to overcome capacity bottlenecks inherent to discrete tokens. It introduces multi-codebook quantization, which significantly enhances token expressiveness and enables UniTok to achieve performance superior to or competitive with domain-specific continuous tokenizers.