RESTRUCTURING VECTOR QUANTIZATION WITH THE ROTATION TRICK

Summary from the TLDR AI newsletter

Vector Quantization relies on gradient estimation through what's known as the Straight Through Gradient estimator. However, its direction is sometimes incorrect. This paper suggests rotation as a way to fix the gradients and improve code book utilization.