LLMS FOR RELEVANCE: AUTOMATING HIGH-QUALITY PRODUCT RELEVANCE LABELING IN FLIPKART SEARCH

Summary from the TLDR Data newsletter

Flipkart's Product Analyser trains a relevance labeling model in two stages: SFT on millions of balanced query-product examples with generated reasoning traces, then GRPO alignment using an 8B reward model that scores explanation quality alongside label correctness. The automated NDCG report it produces differs from the manual one by under 1%, at 30% lower cost.