Pebble is a key-value store written in Go that draws inspiration from LevelDB and RocksDB but is specifically designed for performance and integration with CockroachDB. The project inherits RocksDB's file formats while adding extensions such as range deletion tombstones, table-level bloom filters, and updates to the MANIFEST format. Rather than attempting to replicate every RocksDB feature, Pebble deliberately targets the specific use case and feature set required by CockroachDB, implementing capabilities like block-based tables, checkpoints, indexed batches, level-based compaction, merge operators, prefix bloom filters, range deletion tombstones, reverse iteration, SSTable ingestion, snapshots, and single delete operations.
Pebble was introduced as an alternative storage engine in CockroachDB v20.1 in May 2020 and became the default storage engine in CockroachDB v20.2 in November 2020. The project is production-ready and has been deployed at scale by CockroachDB users. According to GitGenius activity tracking, the repository has processed 1199 issues and pull requests with a median response latency of 55.1 hours, though the mean latency of 4960.3 hours reflects occasional longer-term discussions on complex issues. The most active labels tracked are T-storage with 1089 events and A-storage with 1087 events, indicating heavy focus on storage layer concerns. RaduBerinde leads contributor activity with 814 tracked events, followed by jbowens with 698 events and cockroach-teamcity with 692 events.
The database implements several performance improvements over RocksDB. These include faster reverse iteration through backwards links in the memtable's skiplist, a faster commit pipeline achieving better concurrency, seamless merged iteration of indexed batches, L0 sublevels and flush splitting for concurrent compactions, faster LSM edits through copy-on-write B-trees for file metadata, delete-only compactions that drop entire sstables within range deletion bounds, block-property collectors and filters enabling iterators to skip irrelevant tables and blocks, a range keys API allowing key-value pairs defined over keyspace ranges, and a smaller, more approachable codebase than RocksDB.
Pebble v1 maintains forward compatibility with RocksDB 6.2.1, meaning databases generated by RocksDB can be upgraded for use with Pebble. However, Pebble v2 and newer versions do not support opening RocksDB-generated databases. The project uses format major versions to manage backwards-incompatible changes, with 19 distinct format versions documented from FormatMostCompatible through FormatColumnarBlocks. Different CockroachDB releases support different ranges of these format versions, with migrations either occurring in the background or blocking until completion. The repository explicitly warns that Pebble may silently corrupt data if used with RocksDB databases employing unsupported features, and it does not implement numerous RocksDB capabilities including backups, column families, FIFO compaction, hash table formats, memtable bloom filters, persistent caches, plain table formats, sub-compactions, transactions, and universal compaction styles.
The codebase is classified across multiple storage-related categories including LSM-tree architecture, log-structured storage, sstables, compaction, transaction support, and memory management. The project maintains nightly benchmarks and provides comprehensive documentation on implementation differences from RocksDB, making it accessible for developers integrating it into their systems.
