Geth infinite sync issue (or pending state entries never ~0)

Question

I am running a geth node:

geth --cache=1024 --fast

Disk Info:

=== START OF INFORMATION SECTION ===
Vendor:               VMware
Product:              Virtual disk
Revision:             2.0
Compliance:           SPC-4
User Capacity:        214,748,364,800 bytes [214 GB]
Logical block size:   512 bytes
LB provisioning type: unreported, LBPME=-1, LBPRZ=0
Device type:          disk
Local Time is:        Wed Jul  4 17:34:27 2018 CEST
SMART support is:     Unavailable - device lacks SMART capability.

Memory info :

8 cores
12 gb

Now coming to the output of geth:

eth.syncing command

{
  currentBlock: 5904361,
  highestBlock: 5904527,
  knownStates: 10990364,
  pulledStates: 10963864,
  startingBlock: 5904213
}

eth.getBlock("latest").number

0

eth.getBlock("pending").number

1

Here is one of my output logs:

INFO [07-04|17:24:35] Imported new state entries 
count=635  elapsed=1.821ms   processed=9865187 pending=24195  retry=0
duplicate=882 unexpected=3966`

Please focus on pending value 24195. This value always keeps changing between 20k-40k but never comes to 0

To me it seems like ethereum is updating faster than my node is catching up. It is like Ethereum network is always one step ahead in the trie and my node is never able to catch up.

If anyone has faced this situation or has expert knowledge, suggest what to do in such a scenario?

Answer 1

My node is now fully synced in 6 hours. I used parity to sync the node with --geth option. No SSD was needed, working well in RAID:

Command:

nohup parity --mode active --tracing off --pruning fast 
--db-compaction ssd --cache-size 3072 --geth &

--mode

Used: "active"

Explanation : Set the operating mode. MODE can be one of: last - Uses the last-used mode, active if none; active - Parity continuously syncs the chain; passive - Parity syncs initially, then sleeps and wakes regularly to resync; dark - Parity syncs only when the RPC is active; offline - Parity doesn't sync. (default: last)

--tracing

Used: off

Explanation: Indicates if full transaction tracing should be enabled. Works only if client had been fully synced with tracing enabled. BOOL may be one of auto, on, off. auto uses last used value of this option (off if it does not exist). (default: auto)

--pruning

Used: Fast

Explanation: Configure pruning of the state/storage trie. METHOD may be one of auto, archive, fast: archive - keep all state trie data. No pruning. fast - maintain journal overlay. Fast but 50MB used. auto - use the method most recently synced or default to fast if none synced. (default: auto)

--db-compaction

used: ssd

Explanation: Database compaction type. TYPE may be one of: ssd - suitable for SSDs and fast HDDs; hdd - suitable for slow HDDs; auto - determine automatically. (default: auto)

Note: Server has HDD RAID disk which is quite close to SSD in writes but much much slower (100x slower) in reads. So I think I fooled parity in believing it is on SSD, don't try it on HDD (not sure what will happen )

--cache-size

Used: 3072

Explanation: Override database cache size. (default: 128)

Note: I have huge server so I can afford to use 3GB cache. Although cache size seems to boost sync exponentially , be careful with the number.

--geth

Explanation: Run in Geth-compatibility mode. Sets the IPC path to be the same as Geth's. Overrides the --ipc-path and --ipcpath options. Alters RPCs to reflect Geth bugs. Includes the personal_ RPC by default.

Note: geth mode is available on parity, later I used geth attach and ran eth functions.

--warp

Explanation: Does nothing; warp sync is enabled by default.

Note: Warp is default for Parity , it is amazing feature which syncs Ethereum states fast.

References: https://wiki.parity.io/Configuring-Parity

Final note: I don't understand why people are still using Geth, Parity synced entire blockchain in 6-8 hours while Geth could never achieve it (Node was running for weeks). Maybe I was missing something, thoughts are welcome.