Pebble is a key-value store written in Go that draws inspiration from LevelDB and RocksDB but is specifically designed for performance and integration with CockroachDB. The project inherits RocksDB's file formats while adding extensions such as range deletion tombstones, table-level bloom filters, and updates to the MANIFEST format. Rather than attempting to replicate every RocksDB feature, Pebble deliberately targets the specific use case and feature set required by CockroachDB, implementing capabilities like block-based tables, checkpoints, indexed batches, level-based compaction, merge operators, prefix bloom filters, range deletion tombstones, reverse iteration, SSTable ingestion, snapshots, and single delete operations.
Pebble was introduced as an alternative storage engine in CockroachDB v20.1 in May 2020 and became the default storage engine in CockroachDB v20.2 in November 2020. The project is production-ready and has been deployed at scale by CockroachDB users.
The database implements several performance improvements over RocksDB.
Pebble v1 maintains forward compatibility with RocksDB 6.2.1, meaning databases generated by RocksDB can be upgraded for use with Pebble. However, Pebble v2 and newer versions do not support opening RocksDB-generated databases. The project uses format major versions to manage backwards-incompatible changes, with 19 distinct format versions documented from FormatMostCompatible through FormatColumnarBlocks. Different CockroachDB releases support different ranges of these format versions, with migrations either occurring in the background or blocking until completion. The repository explicitly warns that Pebble may silently corrupt data if used with RocksDB databases employing unsupported features, and it does not implement numerous RocksDB capabilities including backups, column families, FIFO compaction, hash table formats, memtable bloom filters, persistent caches, plain table formats, sub-compactions, transactions, and universal compaction styles.
The codebase is classified across multiple storage-related categories including LSM-tree architecture, log-structured storage, sstables, compaction, transaction support, and memory management. The project maintains nightly benchmarks and provides comprehensive documentation on implementation differences from RocksDB, making it accessible for developers integrating it into their systems.
