The Stack v2 is a dataset curation framework for large-scale code training data.
The project addresses the challenge of preparing high-quality code datasets for training large language models. It provides the tooling and methodology used to curate The Stack v2, a foundational dataset that powers StarCoder2 and similar code generation models. The approach involves filtering, deduplication, and quality assessment of source code collected from public repositories, with the curation logic implemented primarily through Jupyter notebooks that document the data processing pipeline.
Developers working on code language models or those needing to construct their own code training datasets will find this project most relevant. It suits teams building or fine-tuning code-focused models who want to understand or replicate the curation practices behind a production-scale dataset. The project is particularly valuable for researchers studying data quality's impact on model performance, as it exposes the specific filtering and deduplication strategies applied during dataset construction.
The project maintains active development with regular updates to the curation pipeline and dataset processing logic. Work continues on refining filtering heuristics and expanding the dataset's language and domain coverage. The codebase receives ongoing maintenance to address edge cases discovered during large-scale data processing. Documentation through executable notebooks enables reproducibility and allows other teams to adapt the curation methodology to their own datasets.
