SCALING MODEL EVALUATION AT PINTEREST WITH RAY (PART 4)

Summary from the TLDR Data newsletter

Pinterest used to run inference with Ray, write predictions to S3, then start a Spark job to read them back and compute metrics. Its new evaluation platform computes partial metrics batch by batch inside the Ray inference job and combines them at the end. That cut end-to-end evaluation time by up to 32% for feature-importance runs.