THE SECRET FOR BETTER A/B RESULTS (THAT MANY EXPERTS DON'T EVEN GET RIGHT)

Summary from the TLDR Marketing newsletter

A/B test accuracy heavily depends on ensuring users actually see the variation being tested. Many testers make the mistake of including users in results who never saw the changed element, especially if it appears below the fold or in navigation. The solution is to track element visibility, such as requiring 50% visibility for at least 2 seconds, using custom JavaScript or advanced testing tools. Re-running past tests with visibility triggers can reveal winning variations that previously showed flat results.