HOW TO WRITE A GREAT EXPERIMENT PLAN

Summary from the TLDR Marketing newsletter

An experiment plan should prevent costly errors, not serve as a ceremony. Start with an evidence-based hypothesis and one primary metric, then size the test around the smallest change worth shipping. Count only users who reached the change, since including everyone assigned dilutes the effect. Before launch, agree on stopping thresholds and on what happens under each outcome, so that early peeks don't drive the decision. After launch, confirm the traffic split holds.