EVALUATION QUIRKS, METRIC PITFALLS, AND SOME RECOMMENDATIONS

Summary from the TLDR Web Dev newsletter

There are a few issues with system evaluation practices, like in cases where metrics with similar names yield vastly different results, implementation bugs lead to inflated scores, and incorrect predictions improve the score. Developers need to be careful when choosing metrics and understand what a metric truly measures to make sure it's implemented correctly.