0

I just built myself a hardware full node at home. It's running geth and I'm accessing the on-chain data through the JSON RPC API. I'm using web3.py because of convenience.

I want to have a "ledger" basically of all the transactions that I sent or received to a specific address. This post already detailed how to do this. The only issue is, that my account is kinda old. The script I'm running is able to process aprox 1.000 blocks within 3 minutes. If we extrapolate a little bit here, 1.000.000 blocks would roughly take 2 days. 10.000.000 blocks would roughly take 20 days. Blockheight 6.000.000 as an example is dated Jul-20-2018. So even if my script ran 20 days non-stop, I still wouldn't have found all the transactions associated with the account. Is there anything I can do to speed this up? I'm kinda dumbfounded because I thought there was a more efficient way to do this.

Edit1: I should mention that I obviously know about etherscan etc., but the idea of spinning up my own node was to have my own server to serve requests at any point. I want to play around locally, without any 3rd party provider. I hope this works somehow...

Just for completion, here's the function I'm calling:

def getAllTransactionsOfAddress(address):
    transactionCount = w3.eth.getTransactionCount(address,currentBlock)
    addressBalance = w3.eth.getBalance(address,currentBlock)
    i = currentBlock
    while i >= 0 and (transactionCount > 0 or addressBalance > 0):
        try:
            block = w3.eth.get_block(i, True)
            if block and block.transactions:
                for transaction in block.transactions:
                    if address == transaction['from']:
                        if transaction['from'] != transaction.to:
                            addressBalance = addressBalance + transaction.value
                        print(transaction.blockHash,transaction.hash,transaction['from'],transaction.to)
                        transactionCount -= transactionCount
                    if address == transaction.to:
                        if transaction['from'] != transaction.to:
                            addressBalance = addressBalance - transaction.value
                        print(transaction.blockHash,transaction.hash,transaction['from'],transaction.to)
        except:
            print("Error in block"+i)
        i-=1
        if i % 1000 == 0:
            print(str(i) + " " + str(datetime.now().strftime('%Y-%m-%d %H:%M:%S')))

Thanks and Cheers!

2
  • If you want the fastest way and don't care about the cost. I would recommend Google BigQuery. Likely instant result, much faster than looking up your own node.
    – minhhn2910
    Commented Jan 26, 2023 at 2:07
  • Thank you for your answer =) I would make a note of this answer as a last resort if there is no other way. Because then it would feel like setting up my own node was kind of pointless. Maybe one day I'll become a validator myself, but until then I was going to play around with the web3.py library, assuming I'd be able to write programs/scripts using the node as my data source. But if it's too slow to serve those needs, I'd be a bit disappointed.
    – PeepoXBT
    Commented Jan 26, 2023 at 9:51

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.