StarCoder2 is a family of code generation models developed by the BigCode project, consisting of three sizes: 3 billion, 7 billion, and 15 billion parameters. The 3B and 7B variants were trained on more than 3 trillion tokens, while the 15B model underwent training on over 4 trillion tokens. The architecture incorporates Grouped Query Attention with a context window of 16,384 tokens and sliding window attention of 4,096 tokens.
The repository serves as the primary hub for StarCoder2 development and documentation. The codebase is primarily written in Python and includes comprehensive documentation for model deployment, fine-tuning, and evaluation workflows.
The quickstart section provides installation instructions and demonstrates model usage across different hardware configurations, including CPU, GPU, and multi-GPU setups. The documentation covers running models in full precision, using torch.bfloat16 for reduced memory consumption, and quantized versions through bitsandbytes at 8-bit precision. The repository explicitly notes that StarCoder2 models are designed for code completion tasks rather than instruction-following, meaning they perform better with code context than with natural language commands.
Fine-tuning capabilities are a central feature of the repository. The documentation outlines a complete setup process including PyTorch installation and dependency management through requirements files. The fine-tuning approach leverages the PEFT library for Low-Rank Adaptation training combined with bitsandbytes for 4-bit quantization, using the SFTTrainer from the TRL library. An example demonstrates fine-tuning StarCoder2-3B on Rust code from the-stack-smol dataset, with guidance for adapting the process to other programming languages and datasets.
Evaluation resources are integrated into the repository workflow, directing users to the BigCode-Evaluation-Harness for assessing code language model performance and the BigCode Leaderboard for comparative benchmarking. The repository includes references to related projects including the original StarCoder repository and SantaCoder-Finetuning for additional fine-tuning resources.
Community engagement around StarCoder2 shows measured activity levels.
