SPADE is a semantic image synthesis framework developed by NVIDIA Labs that generates photorealistic images from semantic layout maps. The repository implements the Spatially-Adaptive Normalization technique, a novel normalization layer designed to preserve semantic information during image generation. The work was published at CVPR 2019 as an oral presentation by Taesung Park, Ming-Yu Liu, Ting-Chun Wang, and Jun-Yan Zhu.
The core contribution of SPADE is its normalization approach, which addresses a key limitation in conditional image generation where standard batch normalization can wash away semantic information from input layouts. By using spatially-adaptive parameters that vary across spatial locations based on the semantic layout, SPADE enables the generation of high-quality, semantically-consistent images. The framework supports training and inference on multiple datasets including COCO-Stuff, Cityscapes, and ADE20K, with pretrained models available for download.
The repository is built on PyTorch 1.0 and Python 3, requiring significant computational resources for reproduction of published results, specifically an NVIDIA DGX1 machine with eight V100 GPUs. The codebase is organized into modular components including training and testing entry points, trainer classes, network architecture definitions, and dataset loaders. A dynamic options system using argparse allows flexible configuration of models, networks, and datasets, with individual classes able to modify command-line options based on their specific requirements.
SPADE supports both standard conditional generation and VAE-style training with an image encoder for multi-modal outputs and style control. The VAE variant introduces a netE encoder alongside the generator and discriminator, trained with KL-Divergence loss to enable diverse image generation from the same semantic layout. Dataset preparation is flexible, supporting both standard benchmark datasets and custom datasets through a custom dataset loader that requires specification of label directories, image directories, and metadata about label classes and instance maps.
The repository includes GauGAN, an interactive application for generating realistic landscape images from user-drawn layouts. While the online demo is available through NVIDIA's AI Playground, the model weights for GauGAN are not released in this repository. The project page, paper, and multiple demonstration videos are linked in the README, providing comprehensive documentation of the method and its applications.
According to GitGenius activity tracking, the repository shows minimal recent engagement with a median issue and pull request response latency of 0.0 hours across two tracked items, though the mean latency is significantly higher at 14302.9 hours, suggesting sporadic responses. The most active contributors tracked are Richarn290 and dongyanlin, each with one recorded event. The repository shares contributors with major open-source projects including Microsoft's VSCode and TypeScript implementations, as well as the Rust language repository.
The code is released under the CC BY-NC-SA 4.0 license for academic research use only, with commercial inquiries directed to NVIDIA's research department. The README notes that a more performant reimplementation is available in the separate Imaginaire repository, indicating that while this original SPADE implementation remains available, NVIDIA has developed an improved version for production use.
