I have to scan a large amounts of old block. I using subcribe newBlockHeaders so I have new_block.number (2s/block). Then I find in db the latest block number (db_block.number). I use

for(i=db_block.number ; i<db_block.number+200 ; i++){
    web3.eth.getBlock(i, true, function (error, result) {
        //do somethings 

then insert blocknumber to db, db_block.number will increase every time for loop is finish and for loop run again until db_block.number == db_block.number. But after some for loop, its return message 'out of memory'. I know this error because when the loop is finish, my 'do somethings' code is not finish yet. So how can I know when all for loop had finish to start a new one?

  • This seems unrelated to Ethereum, the problem appear from mixing sync code (for loop) with async code (gethBlock callback). One way to achieve that in javascript is to use chain promises. – Ismael Apr 18 at 3:13

Sounds like you are not going to fit all block data in the memory once. Ethereum unpacked state is hundreds of gigabytes at the moment.

Instead, store the blocks on a database instead of memory if you need to do any full scans.

Or configure your Node to have more memory if you think all data you are processing is going to fit in RAM.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.