8

I am trying to traverse over blocks and get their transaction information like this:

for (var i = 0; i < 1000; i++) {
      var block = web3.eth.getBlock(i);
      console.log( block.number);
}

Problem: I am able to fetch first (525-545)th blocks and then, web3.eth.getBlock(i) call halts the execution and loop stuck.

It is not about the incoming block because when I start the loop from the block 500th block again it correctly fetches first 500 blocks and halts again.

The interesting thing is, iteration count over loop is not stable. It is always changing between 525-545 and pauses in this range.

I tried to put a delay between requests but that also didn't work.

geth version 1.6.1-stable-021c3c28 and web3.js version ^0.18.4

Is there a limit(such as RPC limit on macOS) for API calls? How could I fix this issue.

Please note that: This issue works perfectly fine on Linux but does not work on macOS.

  • post this issue on github i think it's due to the macos impelemntation – Sig Touri Jun 7 '17 at 13:11
  • This is not related to implementation, it should be related to some other factors. @ Sig Touri – alper Jun 7 '17 at 15:31
  • Are you using the RPC or the IPC provider? I'm assuming RPC, because you're using synchronous calls... so maybe there's some sort of limit on file handles that you're running into? – DeviateFish Jun 8 '17 at 20:15
  • The way I run it: geth --targetgaslimit "994712388" --port 3005 --networkid 23422 --identity node1 --verbosity 3 --nat none --datadir="$datapath" --rpc --rpcaddr="localhost" --rpcport 8545 --rpccorsdomain="*" --maxpeers=512 --rpcapi "admin,eth,net,web3,debug" @DeviateFish – alper Jun 8 '17 at 21:26
  • I'm wondering if you're running into file descriptor exhaustion or something of the like that's specific to the osx unix-like environment. I'll try to get the system set up on a mac on my end do some digging. – DeviateFish Jun 8 '17 at 22:04
2
+50

I don't know why it behaves like that. Maybe synchronous calls are overwhelming the node? I tried code below with Metamask and production blockchains and it seems to work.

  function getBlock(number) {
    web3.eth.getBlock(number, function(error, block) {
      console.log("Next block:"+number);
      if(error)
        console.log(error);
      else {
        console.log(block);
        if(number < 1000) {
          getBlock(number+1);
        }
      }
    });
  };
  getBlock(0);
  • Do you have a mac?@Lukasz Zuchowski – alper Jun 7 '17 at 16:39
  • I do. I will make that test soon with the testnet and let you know. – Lukasz Zuchowski Jun 7 '17 at 16:40
  • I tested it in Chrome with Metamask on privatenet with geth in version Version: 1.6.1-stable and it seems to work (the code above). Now it's on block 670 and still working fine. – Lukasz Zuchowski Jun 7 '17 at 16:50
  • Wow it worked for me as well and way faster than mine. Do you thing what is the reason wrong with my code and yours is very fast and working? @Lukasz Zuchowski – alper Jun 7 '17 at 16:56
  • 1
    @Avatar Hard to guess why it's much faster because I don't know the implementation details in the geth/web3 but... It's async and yours was synchronous. Please note that the synch calls are going to be removed in the web3 1.x. So maybe under the hood the synchronous call are made in asynchronous way (web3/geth is lunching asynchronous operation and wait until it's over checking from time to time if it's over or not). Asynch call probably just add a callback on the end of the execution, so there is no need to check if it's done with some discrete interval. But that just my hypothesis. – Lukasz Zuchowski Jun 7 '17 at 17:05
2

the problem is that you are not doing it asynchronously. Firing the 500 calls the sync way lead to an overload. You need to

  1. Do all calls async

    promisifiedGetBlock = function (theBlock) {
    return new Promise(function (resolve, reject) {
        web3.eth.getBlock(theBlock, function (e, result) {
            if (e !== null) {
                reject(e);
            } else {
                resolve(result);
            }
        });
    }); };
    
  2. Limit the concurrency of created promises ("Only X calls the same time and wait for them to resolve before firing the next bunch") which can be done with bluebird, e.g. example for concurrency limit.

Always remember, the node client behaves like a dumb machine - press a button too often, and it will stop responding.

0

Probably what you need is to add a callback and do the processing inside

web3.eth.getBlock(i, function(error, block){
  if(!error)
    console.log(block.number)
  else
    console.error(error);
})
  • did not fix the problem. @webjunkie – alper Jun 1 '17 at 21:01
0

Why don't you better subscribe to an event and run a callback function? Here is how I would map React.js state to a new block re-render page:

const filter = web3.eth.filter('latest');
filter.watch((error, result) =>{
  const block = web3.eth.getBlock(result, true);
});

You can basically pass a callback to a filter function: https://github.com/ethereum/wiki/wiki/JavaScript-API#web3ethfilter

From: How to map a web3 to a react.js state

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.