TLDR Data · MISCELLANEOUS

WHY SPOTIFY IS NOT USING BAYESIAN A/B TESTING

engineering.atspotify.com12 min read

Summary from the TLDR Data newsletter

Bayesian experimentation is not one method but a choice of prior, likelihood, and stopping rule. Spotify shows that common flat-prior posterior thresholds can reproduce frequentist peeking, while Bayes factors can control false positives under optional stopping. Choose guarantees for the experiment program first, then the inference configuration.