The meta-llama/llama repository is a Python-based inference codebase for Llama language models that has been deprecated as of the Llama 3.1 release. The repository originally served as a minimal example for loading and running inference on Llama 2 models, which ranged from 7 billion to 70 billion parameters in size. The codebase included both pre-trained base models and fine-tuned chat variants designed for dialogue applications.
The repository's primary functionality centered on model inference with specific requirements for different model sizes. The 7B model required a model-parallel value of 1, the 13B model required 2, and the 70B model required 8. All models supported sequence lengths up to 4096 tokens, with users able to configure max_seq_len and max_batch_size parameters according to their hardware constraints. The codebase included example scripts such as example_text_completion.py for pretrained models and example_chat_completion.py for fine-tuned chat models, with the latter requiring specific formatting including INST and SYS tags, BOS and EOS tokens, and particular whitespace conventions.
Model access was controlled through a registration process on Meta's website, where users would receive signed download URLs valid for 24 hours with download limits. The repository also provided access through Hugging Face, where users could request access by acknowledging the license in model cards. The download.sh script automated the model retrieval process, requiring wget and md5sum to be installed on the user's system.
The repository's contributor network overlapped with major projects including microsoft/vscode, microsoft/typescript, and rust-lang/rust.
The repository is now deprecated in favor of a consolidated ecosystem of specialized repositories. Users are directed to llama-models for foundation models and utilities, PurpleLlama for safety and inference mitigations, llama-toolchain for model development interfaces, llama-agentic-system for end-to-end agentic applications, and llama-cookbook for community-driven scripts and integrations. The original Llama v1 release remains available in the llama_v1 branch. The codebase included a Responsible Use Guide addressing potential risks and ethical considerations for developers deploying these models.
