24

Ethereum clients (such as geth, eth, pyethapp, ethereumj, and ethereumH) download the Ethereum blockchain from their peers and then store a copy of it locally, presumably indexed within some sort of database for efficient lookup and retrieval. Other uses for databases within the clients may include storage of the state, tracking of peers, etc.

Which databases are used by the respective clients, and for what reasons have they been selected?

  • 2
    This sounds very broad. Maybe you are interested in the different approaches how the blockchain is stored locally? Could you clarify why you want that for all existing clients? – Waqar Lim Feb 2 '16 at 1:24
  • 3
    There are big differences in performance, use cases etc. between leveldb, sql, or whatever. I considered asking 5 different questions, and still can, but since I think a single answer can compile the info it made more sense to put it all into one. – Jeff Coleman Feb 2 '16 at 2:02
  • 1
    For Parity see: ethereum.stackexchange.com/questions/13639/… – eth Mar 15 '18 at 1:44
30

Geth uses leveldb. Some of the reasons behind choosing it was:

  • In process database (i.e. I don't need to run an additional software)
  • Native implementation in Go (i.e. cross platform to everything Go supports)
  • Key/value store (i.e. values are arbitrary binary blobs, no schema constraints)
  • Modern data storage (i.e. multiple layers on disk, organized in the background)
  • Proven track record (i.e. many companies and other databases build on top)

However, there are also limitations:

  • Ethereum uses hashes exclusively for identifying anything. However, hashes are uniformly randomly distributed identifiers. Leveldb keeps keys organized "alphabetically" on disk, so accessing values associated with hashes is very expensive. This is mostly felt for the state trie, which is hundreds of thousands of nodes scattered and pointing all over the disk. However, without a specific database modelling the Ethereum trie by design, I don't see how this could be further optimized.
  • Leveldb was originally designed by Google to be a disk backed memory database. As long as most of it can fit into memory, it performs very well, but as more and more disk access is required, performance can degrade. Facebook's RocksDB tries to address some of these issues + multi threaded background compaction, but its still a fairly new thing and being in C makes portability hard.
4

Parity is using RocksDB as it's database.

From reading here it seems the reason for choosing it was performance. Faster than LevelDB in their tests.

Lately though they have hit some performance issues and a lot of corruption issues with RocksDB and are developing a proprietary DB to better suit Parity, ParityDB. Progress can be tracked in this issue.

0

There is a new Java Ethereum client recently released Pantheon from pegasys. It uses RocksDb see implementation. I have been developing with it, it supports clique and synced it with Rinkby and the database currently on disk its less than <30Gb. Its using the rocksdb sst file format read more here *.sst files on disk the code here shows you how simple and accessible the persistence api is. There are many reasons why you would choose RocksDB but not least for the Pantheon devs would be a well developed Java api

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.