The mistral-inference repository is the official inference library for Mistral AI models, though it is currently archived and no longer actively maintained. The repository contains minimal code required to run various Mistral models, serving as a reference implementation for inference across Mistral's model lineup. The primary language is Jupyter Notebook, reflecting its educational and demonstration-focused nature.
The repository supports a comprehensive range of Mistral models including the 7B base and instruct variants, Mixtral 8x7B and 8x22B mixture-of-experts models, specialized variants like Codestral 22B for coding tasks, Mathstral 7B for mathematical reasoning, Codestral-Mamba 7B, Mistral Nemo, Mistral Large 2, Pixtral 12B for multimodal tasks, and Mistral Small 3.1 24B. Models are available for download directly from Mistral's CDN with MD5 checksums for verification, or alternatively from Hugging Face Hub. The repository includes detailed documentation on model downloading, installation procedures, and usage examples.
Installation requires GPU availability since the library depends on xformers, which itself needs GPU resources for installation. The repository provides both command-line interface tools and Python API access for model inference. CLI commands include mistral-demo for testing model functionality, mistral-chat for interactive conversations, and specialized chat interfaces for domain-specific models like Codestral and Mathstral. Python usage examples demonstrate instruction following, multimodal instruction following with image inputs, function calling capabilities, and fill-in-the-middle code completion features.
The deployment section includes Docker image building instructions using vLLM as the serving framework, with the transformers library used instead of the reference implementation. The repository links to official Mistral AI API access through La Plateforme and cloud provider integrations for production deployment.
The repository is classified across multiple domains including software development tools, scalable computing, AI deployment, machine learning models, inference engines, and API clients. It serves as both a reference implementation and practical tool for deploying Mistral models in various inference scenarios, though users are directed toward official documentation and the Mistral AI Discord community for ongoing support and updates.
