HOW TO IMPROVE SEARCH WITHOUT LOOKING AT QUERIES OR RESULTS
Canva developed a system that used generative AI to create a synthetic dataset of realistic queries and designs. This let the company perform offline evaluation of search algorithm changes. The system was meant for privacy, which meant that user queries were hidden from training. The approach allowed for rapid, reproducible testing of code changes locally, speeding up Canva's development process.