I know that the "internal transactions" are not included in the blockchain and can only be retrieved by replaying the transactions and listening to CALL
, CALLCODE
and other similar opcodes. Both Parity and Geth provide means to do that.
However I am having trouble performing a full sync of Geth / Parity. Geth took over a month to sync, uses ~800 GB and as of yesterday doesn't want to run anymore, failing with Fatal: Error starting protocol stack: missing block number for head header hash
. I was not able to find a solution since resync is not really an option for me. On the other hand I have started syncing Parity about a week ago and the process, fast at first, slowed to a crawl at around block #2393000.
As far as I know I can't use warp / fast sync if I want to get internal transactions. Also, I can't use Infura because it doesn't offer appropriate JSON RPC interface. I could use Etherscan, but that seems like cheating - what is the point of having a decentralized system if you fetch your data from a central endpoint? I might have to use it if everything else fails though, but I would appreciate some pointers. I am sure I am not the only one attempting to do this.
TL;DR: what is the fastest way to synchronize nodes so I can fetch the internal transactions from them? Failing that, what are other options to get this data?
SELFDESTRUCT
! I am not that concerned about the explorer itself, I am using a python app and Neo4j for that. However getting the internal txs from the blockchain is... slow, at least via JSON RPC. Will your explorer be able to get internal txs faster somehow? If so, I would be really curious how...geth
, so it is fast, though I didn't focus on code optimization yet. The genesis block with 8893 accounts takes a few minutes to process, mostly because every INSERT is separate, and lookups are done since I replace addresses withuint
IDs, to save on space and increase performance. I have worked with Neo4j , it is fast, but there is no point in exporting data from one NoSQL format (LevelDB) to another NoSQL format like Neo4j. The key is to export it into SQL because it is much flexible than NoSQL.