HOW MUCH GPU MEMORY IS NEEDED TO SERVE A LARGE LANGUAGE MODEL (LLM)?

Summary from the TLDR Web Dev newsletter

A popular LLM interview question is to calculate the GPU memory needed to run inference for different LLMs - this can be done with a simple formula.