Grounded SAM 2 is a foundation model pipeline that grounds and tracks objects in videos by combining Grounding DINO, Florence-2, and SAM 2.
The tool addresses the problem of identifying and following arbitrary objects across video frames without requiring task-specific training. It works by chaining together open-world models: Grounding DINO or its variants detect and localize objects based on text descriptions, Florence-2 provides dense region understanding, and SAM 2 segments and tracks those objects frame-to-frame. The pipeline supports both single-image segmentation and video object tracking with visualization powered by the supervision library.
Developers working on video analysis, object tracking, or visual grounding tasks should consider this tool if they need to detect and follow arbitrary classes without retraining. It suits projects requiring flexible, text-prompted object detection combined with precise segmentation and temporal tracking. The repository emphasizes simplified implementations compared to the original Grounded SAM work, prioritizing ease of use over methodological novelty. Support extends across multiple detection backends including Grounding DINO 1.5, DINO-X, and Florence-2, allowing users to choose based on their accuracy and latency requirements. The tool handles high-resolution imagery through SAHI tiled inference and can track new objects introduced mid-video.
The project maintains active development with regular updates to support new model versions and API changes. Recent work has focused on integrating newer model variants and addressing infrastructure changes in upstream dependencies. The codebase is structured around Jupyter notebooks for demonstrations, making it accessible for experimentation and integration into larger pipelines.
