I just built myself a hardware full node at home. It's running geth and I'm accessing the on-chain data through the JSON RPC API. I'm using web3.py because of convenience.

I want to have a "ledger" basically of all the transactions that I sent or received to a specific address. This post already detailed how to do this. The only issue is, that my account is kinda old. The script I'm running is able to process aprox 1.000 blocks within 3 minutes. If we extrapolate a little bit here, 1.000.000 blocks would roughly take 2 days. 10.000.000 blocks would roughly take 20 days. Blockheight 6.000.000 as an example is dated Jul-20-2018. So even if my script ran 20 days non-stop, I still wouldn't have found all the transactions associated with the account. Is there anything I can do to speed this up? I'm kinda dumbfounded because I thought there was a more efficient way to do this.

Edit1: I should mention that I obviously know about etherscan etc., but the idea of spinning up my own node was to have my own server to serve requests at any point. I want to play around locally, without any 3rd party provider. I hope this works somehow...

Just for completion, here's the function I'm calling:

def getAllTransactionsOfAddress(address):
    transactionCount = w3.eth.getTransactionCount(address,currentBlock)
    addressBalance = w3.eth.getBalance(address,currentBlock)
    i = currentBlock
    while i >= 0 and (transactionCount > 0 or addressBalance > 0):
            block = w3.eth.get_block(i, True)
            if block and block.transactions:
                for transaction in block.transactions:
                    if address == transaction['from']:
                        if transaction['from'] != transaction.to:
                            addressBalance = addressBalance + transaction.value
                        transactionCount -= transactionCount
                    if address == transaction.to:
                        if transaction['from'] != transaction.to:
                            addressBalance = addressBalance - transaction.value
            print("Error in block"+i)
        if i % 1000 == 0:
            print(str(i) + " " + str(datetime.now().strftime('%Y-%m-%d %H:%M:%S')))

Thanks and Cheers!

  • If you want the fastest way and don't care about the cost. I would recommend Google BigQuery. Likely instant result, much faster than looking up your own node.
    – minhhn2910
    Jan 26 at 2:07
  • Thank you for your answer =) I would make a note of this answer as a last resort if there is no other way. Because then it would feel like setting up my own node was kind of pointless. Maybe one day I'll become a validator myself, but until then I was going to play around with the web3.py library, assuming I'd be able to write programs/scripts using the node as my data source. But if it's too slow to serve those needs, I'd be a bit disappointed.
    – PeepoXBT
    Jan 26 at 9:51


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.