THIS A/B TEST RESULT WITH 99% CONFIDENCE IS 87% LIKELY TO BE WRONG

Summary from the TLDR Marketing newsletter

A landing page test claiming a 44% lift and 99% confidence likely produced a false positive because the experiment had just 6.9% statistical power instead of the recommended 80%.