Untitled entry

Summary from the TLDR newsletter

Nvidia says that TensorRT-LLM, an open source software library, will double the H100's performance for running inference on leading large language models. The software is now available in early access. It will be integrated into the Nvidia NeMo LLM framework as part of the Nvidia AI Enterprise software suite when it is released to the public next month. TensorRT-LLM will also support Nvidia's A100, L4, L40, L40, and Grace Hopper Superchip GPUs.