Code Llama is Meta's family of large language models specifically designed for code generation and programming tasks, built on top of the Llama 2 architecture. The repository serves as the official inference implementation for these models, providing minimal example code to load and run Code Llama models. The project is written in Python and addresses a critical gap in open-source code generation by offering state-of-the-art performance among publicly available models.
The Code Llama family includes multiple specialized variants to serve different use cases. Foundation models provide general code understanding, Python-specialized variants are optimized for Python development, and instruction-following models are fine-tuned to follow natural language directives for programming tasks. Each variant comes in four sizes: 7B, 13B, 34B, and 70B parameters, allowing developers to choose based on their computational constraints and performance requirements. The 7B model requires approximately 12.55GB of storage, while the 70B variant requires 131GB, reflecting the significant computational investment in larger models.
A distinctive feature of Code Llama is its infilling capability, available in the 7B and 13B variants of both the base and instruction-following models. This allows the model to generate code that fills in gaps given surrounding context, enabling use cases like code completion within existing files. All models were trained on sequences of 16,000 tokens and demonstrate improvements when processing inputs up to 100,000 tokens, supporting large codebases and comprehensive context windows that exceed typical code completion tools.
The repository includes specific inference examples and formatting requirements for different model types. Pretrained models like CodeLlama-7b and CodeLlama-7b-Python are designed to work as natural continuations of prompts rather than instruction followers. The instruction-following variants require specific formatting with INST tags, system prompts, and BOS/EOS tokens, with the 70B instruction model using a distinct turn-based prompt format. The repository provides helper functions like chat_completion() to handle this formatting automatically.
According to GitGenius activity tracking, the repository shows a median issue and pull request response latency of 7.7 hours across sampled items, indicating active maintenance. The most active contributors tracked include jakkal6, 3oooo0, and allenliu88. The repository shares overlapping contributors with related projects including Anthropic's Claude Code, OpenAI's Codex, and Google Cloud Platform's Terraformer, positioning Code Llama within a broader ecosystem of code generation research.
The models incorporate safety mitigations applied during fine-tuning, with Meta providing a Responsible Use Guide and separate channels for reporting safety concerns. The repository emphasizes that code generated by Code Llama may be subject to third-party licenses, including open source licenses. Model weights and tokenizers require acceptance of Meta's license agreement through their official website, with downloads provided via signed URLs that expire after 24 hours. The project is licensed for both research and commercial use, reflecting Meta's commitment to democratizing access to code generation capabilities across different user types and organizational scales.
