What is RocksDB? The Embedded Key-Value Store

RocksDB is one of the most widely used storage systems, yet almost nobody configures it directly. It sits underneath Kafka, MySQL via MyRocks, TiDB, YugabyteDB, Ceph, and the ledger of the overwhelming majority of Solana validators. 

For something this load-bearing, its coverage is surprisingly thin. 

The official documentation makes for excellent reference material, but a poor first introduction, and most other explainers assume prior knowledge on the reader's part, such as what a LSM tree is.

This is the first article in a series that explores the inner workings of RocksDB, starting with the obvious question.

What is RocksDB?

RocksDB is an embeddable, persistent key-value store optimized for fast storage. It takes keys and values as arbitrary byte arrays, keeps them sorted, and stores them durably on disk. 

The single most important thing to understand up front is that RocksDB is a library, and not a server. There is no process to connect to, no port to open, and no query language to learn.

RocksDB integrates with an application and runs within that process, reading and writing files on local disk. This is what makes RocksDB fast—there’s no network hop between application code and the data that it operates on. It’s minimal because it deliberately leaves replication, sharding, and querying to whatever gets built on top.

Simply put, RocksDB is a storage engine—the component that databases like TiDB and YugabyteDB are built around.

Is RocksDB the same as LevelDB?

No, but they originate from the same codebase. RocksDB began in 2012 as a fork of LevelDB, the lightweight key-value store written by Jeff Dean and Sanjay Ghemawat at Google. 

LevelDB was built for modest environments (e.g., a browser’s IndexedDB backend or a single embedded device), and it made various design decisions to match them, including single-threaded compaction, conservative memory use, and little appetite for tuning.

Engineers at Facebook, now Meta, took that foundation and rebuilt it for server workloads. The mandate was to saturate modern hardware while handling datasets far larger than memory under sustained write pressure. 

RocksDB was open-sourced in 2013 and has diverged significantly from LevelDB, shipping multi-threaded compaction, column families, transactions, backups, merge operators, pluggable compaction styles, and a famously long list of tuning options.

LevelDB and RocksDB bear a family resemblance, but treating the two as interchangeable stopped being reasonable a decade ago.

How does RocksDB work?

RocksDB is built on a log-structured merge tree (LSM tree), which is a data structure that trades read simplicity for write throughput. 

Essentially, incoming writes are written to an in-memory buffer called a memtable. The writes are simultaneously appended to a write-ahead log on disk for durability. 

When the memtable is full, it is frozen and flushed to disk as an immutable, sorted file called a Sorted String Table (SST). SST files accumulate into a leveled hierarchy, and a background process called compaction continually merges them, discarding overwritten values and deleted keys, to keep a tidy, sorted, durable structure.

Reads check the memtable first, and then walk down the different levels of SST files. Bloom filters let RocksDB skip files that cannot possibly contain the key, and a block cache keeps hot data in memory, so most reads never touch more than a file or two.

What is the difference between LSM Trees and B-trees?

B-trees, which are the structure that most traditional databases use, update data in place, meaning that random writes are scattered across disk. LSM trees append and batch, meaning large sequential writes, which is what SSDs and high-ingest workloads want. The trade-off here is that a key’s current value could span several files, and compaction, bloom filters, and caching are needed to keep that trade-off bound. 

Every design decision in an LSM tree ultimately negotiates between three pressures:

• Write amplification
• Read amplification
• Space amplification

Improving any two tends to worsen the third. 

This triangle is the single most important mental model for understanding RocksDB tuning. 

What is RocksDB used for?

RocksDB is used wherever an application needs fast, durable, ordered key-value storage on local disk without the overhead of running a separate database server. Concrete examples make the pattern clearer than categories do.

Kafka Streams keeps each processing task’s state—running aggregations, joins, windowed computations—in a local RocksDB store, so that the state can grow beyond memory and survive restarts without adding a network round trip to every lookup.

Meta’s ZippyDB wraps RocksDB in a replication layer, shard management, and configuration services to offer a fully managed distributed key-value store, with a clean division of labor: RocksDB supplies the storage, and everything server-shaped is built around it.

TiDB’s storage layer runs RocksDB on every node as the engine beneath a distributed SQL database, encoding table structure into key prefixes so that scanning a table becomes one contiguous read over sorted keys.

Every Agave-based Solana validator writes the ledger into RocksDB, keyed by slot so that consecutive ledger data sits adjacent on disk.

The last two examples are interesting because keys are stored in sorted order—byte-wise by default, or by a custom comparator—which makes range scans and prefix lookups efficient.

A surprising amount of real-world schema design in RocksDB-backed systems comes down to exploiting that ordering.

Who uses RocksDB?

Beyond the systems above, RocksDB is foundational to any system that needs an embedded, battle-tested, write-optimized storage engine, and teams opt for it because they’re inheriting a decade of Meta’s production hardening rather than writing one from scratch.

Notably,

• Kafka Streams uses it as the default engine for state stores
• Apache Flink offers a RocksDB state backend for checkpointed state too large to hold in memory
• MyRocks embeds it as a MySQL storage engine, developed at Meta to cut storage footprints on some of the largest MySQL fleets in existence
• TiDB’s storage layer, TiKV, uses RocksDB with separate instances for Raft logs and key-value data
• YugabyteDB’s document store, DocDB, uses RocksDB
• Ceph’s BlueStore uses RocksDB for object-storage metadata
• ArangoDB has shipped RocksDB as its default storage engine since version 3.4, and as its only engine since 3.7

RocksDB is primarily used in write-heavy, larger-than-memory workloads on SSDs, a point that we return to later in this article.

How does Solana use RocksDB?

Solana is a high-performance, low-latency Proof of Stake blockchain renowned for its focus on speed, efficiency, and consumer applications. Solana’s ledger lives in RocksDB. The Agave validator client stores the ledger in the Blockstore, a component containing a RocksDB database split across independent, tunable keyspaces. 

Separate column families hold shred data and shred erasure coding (i.e., the raw units of ledger data as they arrive over the network), transaction statuses, address-to-signature indexes, and assorted metadata.

A validator’s write pattern is extreme when compared to most database workloads. That is, a validator must ingest shreds continuously, keyed by slot numbers that increase almost monotonically, at network line rate, forever. An unpruned archive of the ledger well exceeds hundreds of terabytes and grows by tens of terabytes a year, at the very least.

Under leveled compaction, the shred column families generated enough background compaction work that validators began to hit write stalls, which is RocksDB’s built-in mechanism for slowing ingestion when compaction falls behind. 

Around 2021, the shred column families were switched to FIFO compaction, a minimal style that simply deletes the oldest file when the size cap is reached. FIFO is typically unsafe for general workloads. However, since shred keys arrive in near-monotonic slot order, validators could drop the oldest file, as it would hold the oldest slots. This matched the compaction style to the workload shape almost perfectly.

Subsequent optimizations to leveled compaction reduced its I/O amplification to the point where FIFO no longer provided any advantages, so the FIFO path was deprecated in June 2024 and removed that November.

Firedancer, Jump Crypto’s Solana validator client written in C, drops RocksDB entirely in favor of a purpose-built, in-house storage layer.

Whether a from-scratch storage engine can outrun a decade of RocksDB hardening and tuning is an interesting open question in validator engineering.

When is RocksDB the wrong tool?

RocksDB is the wrong tool more often than its ubiquity suggests. The tuning surface is enormous, with 100s of options whose interactions are non-obvious, so mis-tuning is the default outcome rather than the exception. 

Worse, RocksDB’s defaults are reasonable, rather than optimal, so developers need to understand the amplification tradeoffs outlined above to extract real performance gains. Sustained heavy ingest can outpace background compaction, triggering write stalls that tend to occur when the system is at its busiest.

Moreover, RocksDB is a library and not a server. Everything that a typical database server would provide (e.g., replication, sharding, backups, access control, a query layer, operational tooling) must be built.

Workload shape also matters tremendously. 

RocksDB is a row-oriented, point-lookup-and-range-scan engine. Analytical workloads that scan wide swaths of data and aggregate across columns are better served by a columnar store like ClickHouse.

None of this is reason to avoid RocksDB. Rather, it is reason to choose it deliberately. We at Helius evaluated these risks directly when redesigning our archival layer and chose RocksDB anyway because the workload was precisely the shape RocksDB was built for: point lookups and tight range scans over a large, append-heavy dataset. 

Conclusion

RocksDB is an embeddable, persistent, sorted key-value store that trades operational conveniences for raw performance on local disk. It originated from LevelDB, was hardened at Meta for SSDs and multi-core machines, and now sits inside various systems around the world from stream processors and distributed SQL databases to storage clusters and Solana’s ledger.

If working on high-performance RocksDB setups at scale sounds exciting, come build with us.

Solana is racing to become the settlement layer for global finance, and the infrastructure underneath it runs on exactly the machinery this series describes. Solve hard systems problems on some of the best hardware that money can buy, at planetary scale.

We’re hiring across our engineering team. See all of our open roles at helius.dev/careers.