IMPROVING TEXT EMBEDDINGS WITH LARGE LANGUAGE MODELS

Summary from the TLDR AI newsletter

Researchers at Microsoft used synthetic data to train a decoder-only transformer based on Mistral for embeddings. It is best in class. Interestingly, they use a two-step prompting strategy with GPT-4 to generate the synthetic retrieval training data.