Sending hundreds of signed transactions with web3js which succeed but can only find a couple on Etherscan

Question

I am using web3js to send hundreds of transactions to Ropsten (will eventually change to mainnet) and I am getting a tx hash for all of them, which means that supposedly, all the transactions have been registered correctly onto the blockchain. To achieve this, I am incrementing the nonce by 1 for each transaction, so basically the nonce is:

var nonce = web3.eth.getTransactionCount(config.abtoken.owner) + nonceCount++;

I got the above from error: replacement transaction underpriced since I was getting that error, due to the previous transaction not being mined and hence the nonce could not be the same.

However, when I go to Etherscan, I can only find a few of these tx hashes (about 5 or so). I also noticed that during the first run, after I let it run and came back to check, I saw all of them (about 700 transactions).

Am I doing something wrong here? Or I just need to wait more (according to Etherscan it may take up to 1 hour when the transaction won't be mined on the current block).

I would really appreciate your help on this as it baffles me that it might not be working properly when a tx hash is returned.

Thanks in advance!

Answer 1

As long as your nonces are sequential and do not skip, your transactions will be mined eventually. Etherscan will prioritize indexing immediate pending transactions, and show transactions that are further down the nonce sequence at some point, at their discretion. You will have to contact etherscan for the exact mechanics of how they choose when to index a transaction.

Eventually, my guess is whenever they have resources, they will index all the pending transactions that are likely to only be mined several blocks from now.

Note that nodes do not use the same process, and will always keep the entire tx list in their pool, as long as space permits. Your own node will keep rebroadcasting transactions as necessary.

Answer 2

var nonce = web3.eth.getTransactionCount(config.abtoken.owner) + nonceCount++;

Is this real line from your code? If so, there is a problem here, because such line may produce gaps in nonce values. In case first transaction will be mined while you are still running the loop, value returned by web3.eth.getTransactionCount(config.abtoken.owner) will increase while you implicitly assume this value to stay the same during the loop.

So, instead of

nonceCount = 0;
for (...) {
   ...
   var nonce = web3.eth.getTransactionCount(config.abtoken.owner) + nonceCount++;
   ...
}

you should do:

var nonce = web3.eth.getTransactionCount(config.abtoken.owner);
for (...) {
   ...
   nonce++;
}

This will guarantee sequential nonce values.