5

When initial synchronization (i.e., from block 1) is performed with --fast, both blocks and chainstates are downloaded separately. In my current ongoing download (and apparently others) full synchronization is held up not by waiting for block downloads by waiting for state importation and processing.

What are the state database entries and how can I learn how many exist in the current full blockchain? I'm not finding a status on https://ethstats.net or similar sites.

There is some discussion of the chainstate database entries here and here, and previous size here but I'd like to understand this more thoroughly.

UPDATE: It appears these state database entries are the Merkel Patricia state trie that has to be downloaded before the pivot point is defined and geth moves on to a classical synchronization. This state download period is taking many days and I've had to restart it multiple times. The state count appears to always start at zero each time I restart geth.

So my new question is: Does the Merkel state trie download have to be done at one time, or is the state trie being appended from the previous geth run each time I restart geth?

Improve this question

1 Answer 1

Reset to default
3

If I understand the pull request correctly the state size depends on what block you pivot on and is derived recursively as you download the state data. The total number of states downloaded may not correspond with any value available to a normally running node, which may be why I can't find the value on any stats sites, or through the web3 or JSON RPC API.

From a recent comment on an issue describing this stall in downloading the state size the current size is 70+M entries.

From local testing (I'm giving up and switching to parity) it appears that the syncd state is not preserved through a restart. The stats before restart were:

> eth.syncing
{
  currentBlock: 5067438,
  highestBlock: 5067566,
  knownStates: 4058178,
  pulledStates: 4051109,
  startingBlock: 5066315
}

After Restart:

> eth.syncing
{
  currentBlock: 5067572,
  highestBlock: 5067698,
  knownStates: 476,
  pulledStates: 475,
  startingBlock: 5067438
}
Improve this answer
4
  • Excellent answer! Makes sense that the state size varies this way. And very helpful to know that it resets everytime the download starts - so that means you need to allow the full download to run all at once in order to get fully synced! What a pain, but hopefully will be improved on in future iterations.
    – DirtStats
    Commented Feb 12, 2018 at 16:38
  • Thanks, I since found medium.com/@akhounov/roadmap-for-turbo-geth-31cbfb1e72b7 describing work on improving sync performance in particular around managing these state tries. Oh and parity was no better - I could only get geth with syncmode light to work
    – Rattle
    Commented Feb 12, 2018 at 22:41
  • Could it be that eth.syncing.knownStates stats are only resetted after node restart, but the node actually continues where it stopped? The only proof for that is to stop the download of the snapshot somewhere at 35M and restart. If he finishes at <40M, it would been he continues where he stopped. Didn't read yet that someone tried that.
    – ivicaa
    Commented Mar 4, 2018 at 20:23
  • 2
    I've tested this with Rinkeby. While he was importing the states, I stopped the program each time when he was about to reach 3Mil. The last run finished the sync with displaying 1Mil. entries imported. Hence, it looks like he's continuing where he stopped. Would be good if anyone else could confirm this.
    – ivicaa
    Commented Mar 5, 2018 at 13:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.