MODEL OPTIMIZER

Summary from the TLDR DevOps newsletter

NVIDIA has open sourced its Model Optimizer library, which offers techniques like quantization, pruning, and distillation to accelerate deep learning models. The tool supports inputs from Hugging Face, PyTorch, or ONNX and exports optimized checkpoints for deployment frameworks like TensorRT-LLM and vLLM. Users can install the stable release via PyPI or use NVIDIA container images with the software pre-installed.