SAM 3 is a foundation model for promptable segmentation in images and videos. It extends the capabilities of its predecessor by introducing open-vocabulary concept segmentation, allowing users to detect, segment, and track objects using text prompts or visual prompts such as points, boxes, and masks. The model unifies these segmentation tasks into a single architecture that can handle a vastly larger set of open-vocabulary prompts than prior work, achieving performance levels comparable to human annotators on a benchmark containing hundreds of thousands of unique concepts.
The tool addresses the limitation of previous segmentation models that required predefined object categories or manual specification of every instance to segment. SAM 3 solves this by accepting natural language descriptions of concepts alongside traditional visual prompts, enabling exhaustive segmentation of all instances matching a given concept. This approach allows users to specify what they want to segment in flexible ways rather than being constrained to a fixed set of categories.
Developers should adopt this tool if they need to segment arbitrary object categories defined at inference time, whether in static images or video sequences. It suits applications requiring flexible, open-vocabulary segmentation without retraining. The project provides inference code, finetuning capabilities, trained model checkpoints, and example notebooks demonstrating usage patterns.
The project maintains active engagement with its user base, with nearly all open issues originating from external adopters rather than the core team, indicating substantial real-world adoption. Maintainers respond to new issues and pull requests within a day, demonstrating consistent attention to the codebase and community feedback.
