OPTIMUM-NVIDIA

Summary from the TLDR AI newsletter

An update to TensorRT from Nvidia achieves speeds up to 28x faster than baseline. Llama 2 in particular can run at 1,200 tokens per second in benchmarks. The update takes advantage of the new Hopper and Ada chip architectures.