The Segment Anything Model (SAM) is a foundation model developed by Meta AI Research that performs high-quality object segmentation from various input prompts including points, boxes, and other visual cues. The model can generate masks for individual objects based on prompts or automatically produce masks for all objects within an image. SAM was trained on an extensive dataset comprising 11 million images and 1.1 billion masks, enabling strong zero-shot performance across diverse segmentation tasks.
The repository provides complete inference code for running SAM, downloadable trained model checkpoints, and example notebooks demonstrating practical usage. Three model variants are available with different backbone architectures: ViT-H (default), ViT-L, and ViT-B, allowing users to select based on their computational constraints and accuracy requirements. The codebase requires Python 3.8 or higher, PyTorch 1.7 or later, and TorchVision 0.8 or later, with CUDA support strongly recommended for optimal performance.
The repository includes multiple usage pathways for practitioners. Users can employ SAM with specific prompts to generate masks for targeted objects, use automatic mask generation to segment entire images without prompts, or run inference directly from the command line. The project provides example notebooks covering prompt-based segmentation and automatic mask generation workflows. Additionally, SAM's lightweight mask decoder can be exported to ONNX format for deployment in diverse environments, including web browsers, as demonstrated by the interactive demo available at segment-anything.com/demo.
A React-based web application is included in the demo folder, showcasing how to run mask prediction with the exported ONNX model in browser environments with multithreading support. The repository also provides utilities for working with the underlying SA-1B dataset, including methods to load mask annotations stored in JSON format and decode masks from COCO RLE format into binary representations.
The README indicates that development has progressed to Segment Anything Model 2 (SAM 2), which extends the original model to handle both images and videos. SAM 2 employs a transformer architecture with streaming memory for real-time video processing and was trained on the SA-V dataset, described as the largest video segmentation dataset available. The original SAM repository remains actively maintained as a reference implementation, with the model licensed under Apache 2.0 to facilitate research and commercial applications.
