COMBINING UNSUPERVISED TRAINING AND SUPERVISED FINE-TUNING

Summary from the TLDR AI newsletter

This repo challenges the effectiveness of combining supervised fine-tuning and reinforcement learning in training. Using a more advanced model like GPT-4 for initial training can outperform the more complex two-step method.