HOW TO GET REAL GPU UTILIZATION METRICS

Summary from the TLDR AI newsletter

Nvidia-smi shows a measure of GPU utilization but it is the amount of time where at least one kernel is running, not a full measure of GPU usage. This work by Stas shows how you can get actual FLOP usage.