2

I would like to know if it is at all possible to have multiple geth processes share the same chain data.

I currently have multiple nodes (for a private chain) connected to the same file system via NFS and hosting their chain data on that file system. They each have their own database with their own chain data in it, but I would like to have them share the same data so I do not need to keep a separate redundant copy of all the data for each individual node. This represents a large savings in storage space.

The naive approach of assigning all the processes to the same datadir results in one node working and the other nodes being locked out of the database.

I am open to forking repositories and making software modifications if this is plausible but not currently possible, and I appreciate any insight on how to go about this if necessary.

Thanks in advance for any help or advice you can provide.

Edit 1:

  • My intention isn't for all nodes on the network to share the same data set, just for certain clusters of them. My intention would be for all peers in a particular region to share chain data, but I would control multiple regions, and there would be outside users connecting to the network as well.
  • I control and trust all nodes that would be sharing chain data. Other network peers which I do not control would manage their own chain data.
  • The impression I get is that it is unlikely to be an existing feature, but I am still interested in modifying software to make this happen. If this is impossible due to the details of how things like leveldb and geth work, I would like to understand why.
  • check my answer and if that is not your point, please provide more details. – Kaki Master Of Time Nov 26 '18 at 18:36
3

I don't think that is possible, because it will completely break the goal of the ethereum network and may (and should) break the integrity and network validation of transactions and stored data on the blockchain.

Making nodes share the same data, would make it absurd to pass transactions along the network for validation since all nodes have the same shared data, so one node would ad the transactions and the other would just read the same transactions from the same disk space and verify it is the same ( and it is since it is from the start. you are basically comparing 1 to 1 )

  • Thanks for the prompt response. I understand that using shared data would mean the nodes aren't really checking each other, but that doesn't concern me because I control and trust all of them. To be clear, I will not control every node on the network, but I would like for the ones I do control to share the data set if possible. This probably suggests that it's not an existing feature, but I am still interested in how I might be able to make this work for my use case, even if it requires software modification. – Louis Lamia Nov 26 '18 at 18:59
  • If you control and share data between multiple nodes, it won't be more useful than having multiple endpoint for the node's and smart contract API. it will only relieve your node from RPC requests charge, nothing more. In the case that you did get over the lock of the files (which is kinda not reasonable, since chain data is sensitive and concurrent access would break it and render the node useless until resynchronization) you will have only one node fully working while the others are just idle since if a node validates a transaction, it is by default existant in the chain data for all. – Kaki Master Of Time Nov 26 '18 at 19:47
  • But if you still want to do it, you will have to modify source code, that is clear. – Kaki Master Of Time Nov 26 '18 at 19:47
0

When a node is running it locks the database to protect the integrity of the data. Every time a new entry is added to the database the previous data is invalidated and the new updated state has to be changed. The only way this could work is if the database was accessed as read only and not being changed.

If you wanted to access different parts of the database for each node (similar to sharding) you would have to be able to fragment the data into different parts which seems impossible in the current structure. You need the whole trie in order to get one accounts' information.

In order to make a solution to this problem you have to look at the whole database structure. Since we use a Radix trie seen here: enter image description here

In order to have two different chains with romane and rubens, you'd need to create two duplicate databases, one with a root node of rom and the other with rub. You have now fragmented the trie into two different tries. Then you can have a node working with all requests starting with rom and one with rub.

The process of having two nodes fragmenting the work required is something you'd have to figure out, but the approach would be fragmenting the trie in order to have multiple nodes working on one set of data.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.