BE WARY OF 'META-ANALYSIS' IN DIGITAL EXPERIMENTATION

Summary from the TLDR Marketing newsletter

Meta-analysis combines the results of similar A/B tests into one dataset to identify patterns across many experiments. It can help generate new experiment ideas, but it should not become a source of best practices. Test categories rely on subjective labeling, and different teams can classify the same experiment in very different ways. Most experiments happen under different conditions, so combined results rarely isolate a single variable or reveal the real cause of performance changes.