DISTILLATION FOR BLACK-BOX LLMS

Summary from the TLDR AI newsletter

GAD introduces a new approach for distilling knowledge from black-box LLMs using only generated text, without access to output probabilities. It trains a student model through adversarial learning against a discriminator, enabling on-policy imitation.