How to find the sync status of your geth node via RPC?

Question

NOTE: this question has been marked as a duplicate of another question which is trying to get the status using CONSOLE. We are trying to get the status using the RPC API, which is different. In addition, we have not found a way to get "geth attach" to work (a separate issue), so we need to use the RPC. Also, the RPC is better as we can automate it by making RPC calls to get the status (assuming such a call exists) from a monitoring tool.

We need to work out how long it is going to take a newly created node to complete syncing, and also to monitor its health and restart if necessary.

We can make RPC calls through a tunnel e.g:

{"jsonrpc":"2.0","method":"rpc_modules","params":[],"id":1}

returns:

{"jsonrpc":"2.0","id":1,"result":{"eth":"1.0","net":"1.0","rpc":"1.0","web3":"1.0"}}

We have looked through the list of available API calls for something which returns the current block number or useful statistics and health here and here, and the only thing we could find was

{"jsonrpc": "2.0","method": "eth_blockNumber","params": [], "id": 1}

but this always returns zero:

{"jsonrpc":"2.0","id":1,"result":"0x0"}

even though the log files show there is progress, and it has been running for several hours.

Jun 08 16:42:05 ethd geth[24859]: INFO [06-08|16:42:05] Generating ethash verification cache     epoch=66 percentage=80 elapsed=3.014s
Jun 08 16:42:05 ethd geth[24859]: INFO [06-08|16:42:05] Imported new block receipts              count=391  elapsed=371.395ms number=1949696 hash=063e1b…70018f size=1.68mB   ignored=0
Jun 08 16:42:05 ethd geth[24859]: INFO [06-08|16:42:05] Imported new state entries               count=1199 elapsed=17.478ms  processed=12533838 pending=29750  retry=1   duplicate=807 unexpected=1375
Jun 08 16:42:05 ethd geth[24859]: INFO [06-08|16:42:05] Imported new state entries               count=1096 elapsed=24.295ms  processed=12534934 pending=28917  retry=110 duplicate=807 unexpected=1375
Jun 08 16:42:05 ethd geth[24859]: INFO [06-08|16:42:05] Imported new state entries               count=1333 elapsed=21.002ms  processed=12536267 pending=27709  retry=0   duplicate=807 unexpected=1375
Jun 08 16:42:06 ethd geth[24859]: INFO [06-08|16:42:06] Generated ethash verification cache      epoch=66 elapsed=3.728s
Jun 08 16:42:06 ethd geth[24859]: INFO [06-08|16:42:06] Imported new block headers               count=2048 elapsed=3.895s    number=1951744 hash=b1adc5…7319d9 ignored=0
Jun 08 16:42:06 ethd geth[24859]: INFO [06-08|16:42:06] Imported new state entries               count=1169 elapsed=20.331ms  processed=12537436 pending=26626  retry=0   duplicate=807 unexpected=1375
Jun 08 16:42:06 ethd geth[24859]: INFO [06-08|16:42:06] Imported new state entries               count=1164 elapsed=64.625ms  processed=12538600 pending=25475  retry=0   duplicate=807 unexpected=1375

The command we used to start geth (via systemd service) was:

/usr/bin/geth --syncmode fast --cache=4096 --rpc

============= UPDATE 1 ===========

Below it was kindly suggested to use the eth_getBlockByNumber call, then look for the field called "number". This call returns this info:

{
"jsonrpc": "2.0",
"id": 1,
"result":{
"difficulty": "0x400000000",
"extraData": "0x11bbe8db4e347b4e8c937c1c8370e4b5ed33adb3db69cbdb7a38e1e50b1b82fa",
"gasLimit": "0x1388",
"gasUsed": "0x0",
"hash": "0xd4e56740f876aef8c010b86a40d5f56745a118d0906a34e69aec8c0db1cb8fa3",
"logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
"miner": "0x0000000000000000000000000000000000000000",
"mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
"nonce": "0x0000000000000042",
"number": "0x0",
"parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
"receiptsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
"sha3Uncles": "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
"size": "0x21c",
"stateRoot": "0xd7f8974fb5ac78d9ac099b9ad5018bedc2ce0a72dad1827a1709da30580f0544",
"timestamp": "0x0",
"totalDifficulty": "0x400000000",
"transactions":[],
"transactionsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
"uncles":[]
}
}

And number is still zero, which seems to imply that geth is not working, although the logs imply it is. Any ideas?

=============== UPDATE 2 ===========

I just noticed this call:

{"jsonrpc":"2.0","method":"eth_syncing","params":[],"id":1}

This call, finally, returns non zero information:

{
"jsonrpc": "2.0",
"id": 1,
"result":{
"currentBlock": "0x3c29d6",
"highestBlock": "0x57cdf7",
"knownStates": "0x18b291f",
"pulledStates": "0x18a8cf3",
"startingBlock": "0x0"
}
}

Is this calculation correct?

0x3c29d6 = 3942870
0x57cdf7 = 5754359
(3942870 / 5754359) * 100 = 68%

Could this already be 68%? It has only been running for about 2 hours, so this doesn't seem possible, especially as it has only used about 20GB of Disk so far, and the current fast mode disk size is supposed to be 75GB according to this

Answer 1

 curl -X POST --data '{"jsonrpc":"2.0","method":"eth_getBlockByNumber","params":["latest", false],"id":1}'

then see "number".

Or to have it easier, use

curl -X POST --data '{"jsonrpc":"2.0","method":"eth_syncing","params":[],"id":1}'

It depends on your hard drive how long it will take. If it's a cloud server, you might never catch up if you are unlucky. Go with a machine with a dedicated SSD.

Answer 2

Is this calculation correct?

0x3c29d6 = 3942870 0x57cdf7 = 5754359 (3942870 / 5754359) * 100 = 68% Could this already be 68%? It has only been running for about 2 hours, so this doesn't seem possible, especially as it has only used about 20GB of Disk so far, and the current fast mode disk size is supposed to be 75GB according to this

Yes, looks correct to me. There are more transactions in the later portions of the blockchain, so the later parts use up more disk.

Your percentage shows progress in terms of the number of blocks, but will be misleading in terms of overall progress syncing; as above, there is more to do in the later blocks of ETH mainnet, plus fast sync will slow down towards the end anyway to get states synced.

eth_blockNumber should start to return other than 0 once state syncing gets underway.

Answer 3

I would assume your note pertains to this post about geth attach. I would recommend ensuring that your Geth instance is setup to allow for external connections.

By default, geth will only allow RPC calls originating from the same device as geth is running. You'll need to use --rpcaddr and --rpccorsdomain flags to allow external RPC connections (although this is not advisable from a security standpoint).

A Guide on how to do so is available.