Next to impossible to get a node synced

Question

I have been syncing on a I7-2600 with two WD raptor 10k rpm drives in Raid0 @ 100% disk IO for over a week and only on block 2470271. Another rig I have has 4 drives raid 0 and it has been syncing for 3 days and on block 2425081. I think my drives might fail from the stress. Incredibly frustrating for people who want to support this cryptocurrency but unable to get on board and participate -- especially how the wallet software encourages people to put some ETH in their wallet during the installation process. I suspect some people will never be able to access the money they put into their wallets.

The inefficient manner in which nodes sync and process the chain doesn't make sense. Perhaps someone can explain why this is happening because other cryptocurrency wallet apps I have used (btc/ltc) do not exhibit this behavior syncing. It seems that segment downloaded needs to be processed into the db and this is taxing my HDDs and causes system to be unresponsive.

Since it is possibly to copy the chaindata from one node to another, it would be great to see a P2P download of a large portion of the chaindata db as a seed and then incrementally update. It would speed up syncing 100x!

For someone wanting support this cryptocurrency this has been an extremely frustrating experience.

Answer 1

We have done this recently with geth, with both modes --syncmode full and --sync fast. In any case a SSD is a must (or even better a ramdisk). Also having plenity of memory helps greatly, ie above 8Gb is good, a VM with only 4Gb have problems staying up. For full mode we used 150Gb of disk space, for fast mode around 20Gb, as of 2017-05-31.

Unless you need to use debug api, the fast mode should be enough. It only download the blocks, the transactions receipts and the state, during a weekend it will sync fully.

For the full mode the blocks around 2M-2.5M uses a lots of IO. We've measured times above 40 minutes to sync a single block (with a cheap WD green 1TB disk). Good magnetic disks can give you a much better time, but if you are serious you will ditch them immediately and use SSD or if you can a ramdisk.

Answer 2

You don't mention what wallet/node software you're using, so I'll speak in generalities. Syncing is still kind of a big problem. I'm still not really sure why it performs so badly. I didn't open my wallet for 2 weeks and here I am waiting for 2 days so far to completely sync.

Anyway, your first sync should probably be in fast mode(e.g. geth --syncmode fast). Look into whatever client you have for options. It might even be a benefit for you to blow away your entire data downloaded so far to do it.

Anyway, there's no way your drives should be thrashing due to syncing. There's just not that many operations that can happen. In fact, right now as I sync, my disks are all but idle because the blocks can't come in fast enough. I suspect your RAM might be full up causing your OS to thrash memory into virtual memory/swap and back to cope.

If that's the case, restart your client as you might be stuck on old malicious blocks. That's another reason to fast sync from the start so you're not processing bugged/malicious blocks from early in the chain.