NVIDIA Model Optimizer is a library of model optimization techniques that compresses deep learning models for faster inference across deployment frameworks.
The tool addresses the challenge of reducing model size and latency for production inference by providing a unified interface to state-of-the-art optimization methods including quantization, pruning, neural architecture search, distillation, speculative decoding, and sparsity. Users compose these techniques through Python APIs to generate optimized quantized checkpoints from input models in Hugging Face, PyTorch, or ONNX formats. The library integrates with NVIDIA Megatron-Bridge, Megatron-LM, and Hugging Face Accelerate to support training-time optimization where needed.
The tool is designed for teams deploying large language models and other deep learning models where inference speed matters. It suits projects already committed to the NVIDIA ecosystem, as optimized checkpoints export seamlessly to TensorRT-LLM, TensorRT, vLLM, and SGLang. The unified Hugging Face export API supports both transformers and diffusers models, making it accessible for a range of model architectures. Adoption makes most sense when inference latency is a bottleneck and you have the infrastructure to leverage NVIDIA's deployment frameworks.
The project maintains active development with regular updates to its roadmap and announcement blogs documenting new capabilities. Development activity shows sustained investment in expanding technique coverage and deepening integration with the broader NVIDIA AI software ecosystem.
