YOLACT is a fully convolutional neural network designed for real-time instance segmentation, implementing the methods described in two peer-reviewed papers: the original YOLACT paper and its successor YOLACT++. The repository is written in Python and built on PyTorch, providing researchers and practitioners with a practical implementation for simultaneous object detection and pixel-level mask generation.
The core innovation of YOLACT is its coefficient-based approach to instance segmentation, which enables real-time performance on consumer-grade GPUs. The base YOLACT model achieves 33.5 frames per second on a Titan Xp GPU while reaching 29.8 mean average precision on the COCO test-dev dataset at 550 pixel image resolution. YOLACT++ improves upon this foundation with deformable convolutional layers, achieving 34.1 mAP at the same speed with a ResNet50 backbone, or 34.6 mAP at 27.3 fps using a ResNet101 backbone. The repository provides multiple pre-trained model variants with different backbone architectures including ResNet50, ResNet101, and Darknet53, each with corresponding performance benchmarks.
The implementation supports flexible training configurations on multiple datasets. The primary training dataset is COCO, with the repository including utilities to download and prepare the full dataset and test-dev split. Beyond COCO, the codebase includes configurations for Pascal SBD annotations, enabling rapid experimentation and comparison with other methods. Users can also train on custom datasets by providing COCO-style JSON annotation files and defining dataset configurations in the codebase.
Training infrastructure includes multi-GPU support that automatically scales hyperparameters based on the number of GPUs used, allowing batch sizes to be set as multiples of the GPU count. The training pipeline logs both training and validation metrics by default, with validation mAP computed on the first 5000 images every two epochs. Training can be interrupted at any point, with the system saving checkpoint files at the current iteration. Pre-trained ImageNet weights are required to initialize the backbone networks and are provided through Google Drive links in the documentation.
Evaluation capabilities are comprehensive, supporting quantitative results on COCO, qualitative visualizations, and benchmarking across different configurations. The eval.py script provides extensive functionality accessible through command-line arguments for generating predictions on images and videos, computing metrics, and producing visualizations of segmentation results.
The repository includes detailed installation instructions with options for Anaconda or manual pip-based setup. For YOLACT++, users must compile deformable convolutional layers from DCNv2, requiring a CUDA toolkit installation. The codebase is structured to support both rapid prototyping and production deployment, with clear separation between configuration files, training scripts, and evaluation utilities. Documentation covers custom dataset creation from scratch, providing guidance on annotation requirements and dataset preparation workflows.
