WHY ARE EMBEDDINGS SO CHEAP?

Summary from the TLDR AI newsletter

Embeddings are a fundamental component of every modern retrieval augmented generation system. This post shows what computations are required to produce an embedding and how this can be used to calculate the true dollar cost of processing a token. The cost of processing a token is minuscule, and all embedding models converge to similar semantic representations, so no provider really enjoys a sustainable moat. This results in prices racing to the bottom as the underlying low costs are being passed directly on to consumers.