QUANTIZATION-AWARE DISTILLATION FOR LLMS AND VLMS

Summary from the TLDR AI newsletter

NVIDIA's QAD is a method that uses KL divergence loss to distill full-precision models into quantized students. It enables stable and accurate quantization for complex LLM pipelines without full retraining, recovering near-BF16 accuracy across several Nemotron variants.