MEASURE LESS TO LEARN MORE: USING FEWER, HIGHER-QUALITY METRICS TO CAPTURE WHAT MATTERS

Summary from the TLDR Dev newsletter

Discord reduced its default experiment metrics from approximately 50 to 15, which improved its ability to detect real effects by 45%. By using correlation analysis and PCA to eliminate redundant metrics, the service avoided the stricter thresholds and decreased recall typically caused by multiple comparisons corrections.