DataFlex is a unified benchmark and evaluation platform for data-centric training of large language models that enables dynamic selection, mixing, and reweighting of training data during the optimization process.
The tool addresses the challenge of improving model performance and experimental reproducibility by intelligently scheduling training data. It integrates multiple data-centric techniques—data selection, data mixture, and data reweighting—into a single framework built on top of LLaMA-Factory. Data selection algorithms range from gradient-based methods like LESS and NICE to loss-based and distribution-based approaches, allowing researchers to dynamically choose which samples to train on according to strategies such as focusing on harder examples. Data mixture algorithms adjust the ratio of data from different domains during training. This unified approach consolidates implementations that were previously scattered across difficult-to-reproduce repositories.
Researchers and developers working on large language model training should consider DataFlex if they want to experiment with data-centric optimization strategies without reimplementing multiple algorithms from scratch. The tool suits projects where improving model performance through intelligent data handling is a priority, and where reproducibility across different data selection and mixture techniques matters. The framework integrates seamlessly with LLaMA-Factory, making it accessible to teams already using that training infrastructure. Those seeking to compare different data-centric approaches or reproduce results from published methods will find the unified implementation valuable.
The project maintains active development with support for large-scale model training through gradient computation under DeepSpeed ZeRO-3. The codebase includes reproducible implementations of multiple data selection, mixture, and reweighting algorithms that were previously unavailable or scattered across different sources. Documentation is provided through a dedicated technical site explaining the system's goals and design philosophy.
