Search-R1 is a reinforcement learning framework that trains language models to interleave reasoning with search engine calls.
The framework addresses the challenge of building language models that can both reason through problems and strategically retrieve information from search engines in a coordinated way. Rather than treating reasoning and search as separate stages, Search-R1 trains models to decide when to reason internally and when to call a search engine, learning both behaviors through reinforcement learning. It builds on the veRL framework and extends concepts from DeepSeek-R1 by adding the ability to make interleaved tool calls to search engines, providing an open-source alternative to proprietary systems like OpenAI DeepResearch.
The tool supports multiple reinforcement learning algorithms including PPO, GRPO, and reinforce, works with various base language models such as Llama3 and Qwen2.5, and can integrate with different search backends ranging from local sparse and dense retrievers to online search engines. This flexibility makes it suitable for researchers and teams building tool-augmented reasoning systems who want full control over the training pipeline and model architecture. The framework is particularly relevant for applications requiring factual grounding through search, such as question-answering systems over knowledge bases or real-time information retrieval tasks.
The project maintains active development with multiple experiment logs documenting different training runs and methodology iterations. The codebase includes published research papers detailing the approach and results, alongside trained model checkpoints and datasets made available through standard model repositories. Development activity shows ongoing refinement of the training pipeline and expansion of supported configurations across different RL methods and model families.
