WHY I DON'T TEST DIFFERENT DESIGNS AT THE SAME TIME

Summary from the TLDR Design newsletter

Testing two designs simultaneously creates problems, such as unclear results, contaminated data from participants learning between versions, and requiring three times as many participants. Test one properly-designed version, identify its problems, and fix them rather than conducting comparative evaluations. This approach is presented as more efficient since differences that seem significant to designers often appear identical to users, making comparative testing unnecessary.