HOW NETFLIX MEASURES THE MARKETING YOU CANNOT A/B TEST

Summary from the TLDR Marketing newsletter

Netflix uses geographic holdouts to measure channels that can't be A/B tested at the user level, including TV, OOH, and sponsorships. The approach measures incremental impact but requires careful decisions around test size, timing, media spillover, and carryover. Repeating tests builds stronger evidence, while campaigns without historical data require different methodologies. The challenge is getting leadership to protect test windows and accept the tradeoff between media delivery and measurement certainty.