Zstandard, commonly referred to as zstd, is a fast lossless compression algorithm developed by Facebook and designed for real-time compression scenarios. The reference implementation is provided as an open-source C library dual-licensed under BSD or GPLv2, along with a command-line utility capable of producing and decoding .zst, .gz, .xz, and .lz4 files. The compression format is stable and formally documented in RFC8878, with multiple independent implementations already available across various programming languages.
The algorithm achieves compression ratios comparable to zlib while delivering significantly faster performance. According to benchmarks on the Silesia compression corpus using a Core i7-9700K CPU, zstd 1.5.7 at default settings achieves a compression ratio of 2.896 with compression speeds of 510 MB/s and decompression speeds of 1550 MB/s, outperforming zlib 1.3.1 which achieves 2.743 ratio with 105 MB/s compression and 390 MB/s decompression. The algorithm offers configurable speed-to-compression tradeoffs through negative compression levels specified with the --fast flag, allowing users to prioritize either compression speed or ratio depending on their requirements. Notably, decompression speed remains relatively consistent across all compression settings, a characteristic shared with other LZ-based algorithms like zlib and lzma.
Zstd includes a training mode specifically designed to improve compression of small data sets. This feature addresses the inherent difficulty of compressing small amounts of data by allowing the algorithm to be tuned for specific data types through sample training. Users can create a dictionary from training samples, then use that dictionary during both compression and decompression to dramatically improve compression ratios on small data while simultaneously achieving faster compression and decompression speeds. The dictionary approach is most effective in the first few kilobytes of data, after which the algorithm leverages previously decoded content for improved compression.
The repository supports multiple build systems including Make, CMake, Meson, VCPKG, Conan, Visual Studio, Buck, and Bazel, providing flexibility for integration into various development environments. The Make build system is designated as the reference implementation, with other build systems periodically synchronized to maintain compatibility. The project includes comprehensive testing infrastructure with quick local smoke tests available through make check, and the codebase is continuously fuzzed for security issues through Google's oss-fuzz program.
Zstandard is deployed extensively within Meta and numerous large cloud infrastructures for compressing substantial volumes of data across diverse formats and use cases. The algorithm's combination of fast entropy encoding via the Huff0 and FSE library, configurable compression levels, dictionary support for specialized data types, and broad platform compatibility has established it as a practical solution for modern compression requirements in both streaming and file-based scenarios.
