getTransaction sometimes returns null when transaction is pending

Question

I am looking to get real time transaction data as it is broadcasted to the ethereum network, so I have subscribed to pendingTransactions. This only returns the transaction hash though, so I have to call getTransaction on each hash to get more detailed data.

const Web3 = require('web3')
const web3 = new Web3('ws://localhost:8546');
web3.eth.subscribe('pendingTransactions', function(error, result){
   if (error) console.log(error);
})
.on("data", function(hash){
    let promise = web3.eth.getTransaction(hash);
    promise.then(console.log);
});

It works about 95% of the time, but sometimes it just returns null. I have no idea why this happens. I know this has been asked before here, but I found the answer to be unacceptable. The pending state of the transaction didn't prevent the other 95% from returning data, so why is the 5% having issues? I also tried putting a timeout on the ones that failed and retrying, but it's always the same ones that have issues, so the issue isn't that my cpu might be overloaded.

Is there no reliable way of getting real time transaction data before the transactions are confirmed?

Answer 1

The only reasonable thing I can imagine is that the transaction was replaced by another with a larger fee.

Ethereum clients have a limit on how many transaction they can have in the pending pool at any moment. So transactions with lower fees can be dropped.

Since synchronization it is a p2p protocol some of the low fee transactions will keep appearing and disappearing over time.

Answer 2

Apparently this used to happen on the ropsten network.

I got it today from a light node on ropsten, even though the transaction had 10 confirmations.

It is possible that the getTransaction() request was sent to a peer which was a little bit behind?

Anyway, since this seems to be a rare and temporary issue, my solution has been to use a wrapper function that simply retries.

  async getTransaction(transactionHash: string): Promise<Tx> {
    for (let attempt = 1; attempt <= 60; attempt++) {
      const transaction = await web3.eth.getTransaction(transactionHash);
      if (transaction) return transaction;
      await delay(10 * 1000);
    }
    throw new Error(`Failed to receive transaction from getTransaction("${transactionHash}")`);
  },

For the same reason, I do exactly the same for getTransactionReceipt().