Context
I am trying to build my own Ethereum explorer. I know this already exists on Etherscan out-of-the-box but the features that I want are either rate-limited or not even provided via its API.
To this end, I am trying to implement an address tracker for a specific contract. Particularly, I iterate over all blocks since genesis and for all transactions for a given contract, I keep track of the wallets that interacted with it and try to update their balance accordingly (i.e. +/- the value transferred). The goal is to have a list of all addresses that interacted with that contract and their up-to-date balances.
Resources
I have a local GETH full node (not archive) which is accessible via IPC. I also have rate-limited access to an archive node via JSON RPC. I am exclusively using python 3.9
and web3.py
with version 5.19.0
.
Problem
I find it tricky when trying to extract the number of tokens transferred during an Event. In particular, I don't know apriori whether a custom Event
declared in the ABI (apart from Approval
and Transfer
which are declared in the ERC20 standard) moves tokens or not. What if a custom event (e.g. Lend
or Burn
) exists that is able to change a wallet's balance (other than the standard Transfer
event)? How could I systematically interpret such custom events while having access only to the ABI?
Snippet
A snippet to replicate what I am trying to accomplish is the following. If you only have access to a full-node (and not an archive node) you probably cannot extract the details of that particular transaction because it would be forgotten by your node due to its short memory by the time you read this. To solve this, you can simply select the most recent transaction hash for contract 0x967da4048cD07aB37855c090aAF366e4ce1b9F48
as evinced on Etherscan.io.
import web3
# create a client
w3 = web3.Web3(web3.Web3.HTTPProvider(YOUR_NODE_ADDRESS))
print(f"Connected to Ethereum: {w3.isConnected()}\n")
# get ABI (as found on Etherescan.io)
contract_abi = '[{"constant":true,"inputs":[],"name":"name","outputs":[{"name":"","type":"string"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":false,"inputs":[{"name":"spender","type":"address"},{"name":"value","type":"uint256"}],"name":"approve","outputs":[{"name":"","type":"bool"}],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":true,"inputs":[],"name":"getAccountsLength","outputs":[{"name":"","type":"uint256"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":true,"inputs":[],"name":"totalSupply","outputs":[{"name":"","type":"uint256"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":false,"inputs":[{"name":"_from","type":"address"},{"name":"_to","type":"address"},{"name":"_value","type":"uint256"}],"name":"transferFrom","outputs":[{"name":"","type":"bool"}],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":true,"inputs":[],"name":"decimals","outputs":[{"name":"","type":"uint8"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":true,"inputs":[],"name":"cap","outputs":[{"name":"","type":"uint256"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":false,"inputs":[{"name":"spender","type":"address"},{"name":"addedValue","type":"uint256"}],"name":"increaseAllowance","outputs":[{"name":"success","type":"bool"}],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":false,"inputs":[],"name":"unpause","outputs":[],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":false,"inputs":[{"name":"to","type":"address"},{"name":"value","type":"uint256"}],"name":"mint","outputs":[{"name":"","type":"bool"}],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":false,"inputs":[],"name":"kill","outputs":[],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":true,"inputs":[{"name":"account","type":"address"}],"name":"isPauser","outputs":[{"name":"","type":"bool"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":true,"inputs":[],"name":"paused","outputs":[{"name":"","type":"bool"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":false,"inputs":[],"name":"renouncePauser","outputs":[],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":true,"inputs":[{"name":"owner","type":"address"}],"name":"balanceOf","outputs":[{"name":"","type":"uint256"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":false,"inputs":[],"name":"renounceOwnership","outputs":[],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":false,"inputs":[{"name":"account","type":"address"}],"name":"addPauser","outputs":[],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":false,"inputs":[],"name":"pause","outputs":[],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":true,"inputs":[],"name":"owner","outputs":[{"name":"","type":"address"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":true,"inputs":[],"name":"isOwner","outputs":[{"name":"","type":"bool"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":true,"inputs":[],"name":"symbol","outputs":[{"name":"","type":"string"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":false,"inputs":[{"name":"account","type":"address"}],"name":"addMinter","outputs":[],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":false,"inputs":[],"name":"renounceMinter","outputs":[],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":false,"inputs":[{"name":"spender","type":"address"},{"name":"subtractedValue","type":"uint256"}],"name":"decreaseAllowance","outputs":[{"name":"success","type":"bool"}],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":false,"inputs":[{"name":"_to","type":"address"},{"name":"_value","type":"uint256"}],"name":"transfer","outputs":[{"name":"","type":"bool"}],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":true,"inputs":[{"name":"account","type":"address"}],"name":"isMinter","outputs":[{"name":"","type":"bool"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":true,"inputs":[{"name":"owner","type":"address"},{"name":"spender","type":"address"}],"name":"allowance","outputs":[{"name":"","type":"uint256"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":true,"inputs":[{"name":"_start","type":"uint256"},{"name":"_end","type":"uint256"}],"name":"getAccounts","outputs":[{"name":"","type":"address[]"},{"name":"","type":"uint256[]"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":false,"inputs":[{"name":"newOwner","type":"address"}],"name":"transferOwnership","outputs":[],"payable":false,"stateMutability":"nonpayable","type":"function"},{"inputs":[{"name":"contractOwner","type":"address"}],"payable":false,"stateMutability":"nonpayable","type":"constructor"},{"payable":true,"stateMutability":"payable","type":"fallback"},{"anonymous":false,"inputs":[{"indexed":true,"name":"account","type":"address"}],"name":"MinterAdded","type":"event"},{"anonymous":false,"inputs":[{"indexed":true,"name":"account","type":"address"}],"name":"MinterRemoved","type":"event"},{"anonymous":false,"inputs":[{"indexed":false,"name":"account","type":"address"}],"name":"Paused","type":"event"},{"anonymous":false,"inputs":[{"indexed":false,"name":"account","type":"address"}],"name":"Unpaused","type":"event"},{"anonymous":false,"inputs":[{"indexed":true,"name":"account","type":"address"}],"name":"PauserAdded","type":"event"},{"anonymous":false,"inputs":[{"indexed":true,"name":"account","type":"address"}],"name":"PauserRemoved","type":"event"},{"anonymous":false,"inputs":[{"indexed":true,"name":"from","type":"address"},{"indexed":true,"name":"to","type":"address"},{"indexed":false,"name":"value","type":"uint256"}],"name":"Transfer","type":"event"},{"anonymous":false,"inputs":[{"indexed":true,"name":"owner","type":"address"},{"indexed":true,"name":"spender","type":"address"},{"indexed":false,"name":"value","type":"uint256"}],"name":"Approval","type":"event"},{"anonymous":false,"inputs":[{"indexed":true,"name":"previousOwner","type":"address"},{"indexed":true,"name":"newOwner","type":"address"}],"name":"OwnershipTransferred","type":"event"}]'
# specify contract address
contract_address = '0x967da4048cD07aB37855c090aAF366e4ce1b9F48'
# instantiate the contract
contract = w3.eth.contract(address=contract_address, abi=contract_abi)
# specify a transaction hash
txhash = "0x4cf9e7ddfaf048f8aab2022c51bfb5cbdbc627527c5724b32ff96eb7823fa98d"
# get the transaction
tx = w3.eth.get_transaction(txhash)
print(f"Transaction: {tx}\n")
# get the transaction receipt
receipt = w3.eth.get_transaction_receipt(txhash)
print(f"Receipt: {receipt}\n")
# get the decoded logs
print(f"The logs are: {receipt.logs}\n")
print(f"The decoded Transfer details are: {contract.events.Transfer().processReceipt(receipt)}\n")
The output looks like this:
Connected to Ethereum: True
Transaction: AttributeDict({'hash': HexBytes('0x4cf9e7ddfaf048f8aab2022c51bfb5cbdbc627527c5724b32ff96eb7823fa98d'), 'nonce': 14372, 'blockHash': HexBytes('0xddbc01819cbc461610622b20e9db3b9b04cd83e8e687dc6d65e98fa329ecd09d'), 'blockNumber': 12379948, 'transactionIndex': 4, 'from': '0x33A64dcDfa041bEfebC9161a3e0c6180cd94Fa89', 'to': '0x967da4048cD07aB37855c090aAF366e4ce1b9F48', 'value': 0, 'gasPrice': 80000000000, 'gas': 250000, 'data': '0xa9059cbb000000000000000000000000bb6ac5d29d8da644f41d1ff042652f7ac238127d000000000000000000000000000000000000000000000002b5e3af16b1880000', 'input': '0xa9059cbb000000000000000000000000bb6ac5d29d8da644f41d1ff042652f7ac238127d000000000000000000000000000000000000000000000002b5e3af16b1880000', 'type': '0x00', 'v': 28, 's': HexBytes('0x60658b9e6d418762ea27566d7006de3143636c6802a2c3c823745a9850f1380c'), 'r': HexBytes('0x9361bd1cb63c03dfec52bc46ced34cf8d00f59f04b48e6e294a7b2d28539eaec')})
Receipt: AttributeDict({'blockHash': HexBytes('0xddbc01819cbc461610622b20e9db3b9b04cd83e8e687dc6d65e98fa329ecd09d'), 'blockNumber': 12379948, 'contractAddress': None, 'cumulativeGasUsed': 309482, 'from': '0x33A64dcDfa041bEfebC9161a3e0c6180cd94Fa89', 'gasUsed': 106498, 'logs': [AttributeDict({'address': '0x967da4048cD07aB37855c090aAF366e4ce1b9F48', 'topics': [HexBytes('0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef'), HexBytes('0x00000000000000000000000033a64dcdfa041befebc9161a3e0c6180cd94fa89'), HexBytes('0x000000000000000000000000bb6ac5d29d8da644f41d1ff042652f7ac238127d')], 'data': '0x000000000000000000000000000000000000000000000002b5e3af16b1880000', 'blockNumber': 12379948, 'transactionHash': HexBytes('0x4cf9e7ddfaf048f8aab2022c51bfb5cbdbc627527c5724b32ff96eb7823fa98d'), 'transactionIndex': 4, 'blockHash': HexBytes('0xddbc01819cbc461610622b20e9db3b9b04cd83e8e687dc6d65e98fa329ecd09d'), 'logIndex': 6, 'removed': False})], 'logsBloom': HexBytes('0x00000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000020000000000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000001000000000000000000000200000000000000000000000000000000000000000000000000000000000000000200000000000200000000000000000000000000000002000000008000000000000000000000000040000000000000000000000000000000000000000000000000000001000000000000000000000000000000'), 'status': 1, 'to': '0x967da4048cD07aB37855c090aAF366e4ce1b9F48', 'transactionHash': HexBytes('0x4cf9e7ddfaf048f8aab2022c51bfb5cbdbc627527c5724b32ff96eb7823fa98d'), 'transactionIndex': 4, 'type': '0x0'})
The logs are: [AttributeDict({'address': '0x967da4048cD07aB37855c090aAF366e4ce1b9F48', 'topics': [HexBytes('0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef'), HexBytes('0x00000000000000000000000033a64dcdfa041befebc9161a3e0c6180cd94fa89'), HexBytes('0x000000000000000000000000bb6ac5d29d8da644f41d1ff042652f7ac238127d')], 'data': '0x000000000000000000000000000000000000000000000002b5e3af16b1880000', 'blockNumber': 12379948, 'transactionHash': HexBytes('0x4cf9e7ddfaf048f8aab2022c51bfb5cbdbc627527c5724b32ff96eb7823fa98d'), 'transactionIndex': 4, 'blockHash': HexBytes('0xddbc01819cbc461610622b20e9db3b9b04cd83e8e687dc6d65e98fa329ecd09d'), 'logIndex': 6, 'removed': False})]
The decoded Transfer details are: (AttributeDict({'args': AttributeDict({'from': '0x33A64dcDfa041bEfebC9161a3e0c6180cd94Fa89', 'to': '0xBb6aC5D29D8da644F41d1FF042652f7Ac238127d', 'value': 50000000000000000000}), 'event': 'Transfer', 'logIndex': 6, 'transactionIndex': 4, 'transactionHash': HexBytes('0x4cf9e7ddfaf048f8aab2022c51bfb5cbdbc627527c5724b32ff96eb7823fa98d'), 'address': '0x967da4048cD07aB37855c090aAF366e4ce1b9F48', 'blockHash': HexBytes('0xddbc01819cbc461610622b20e9db3b9b04cd83e8e687dc6d65e98fa329ecd09d'), 'blockNumber': 12379948}),)
On Etherscan.io you can see the following under the particular txhash:
As you can see, the tokens transferred are not shown next to "Value" (because no ETH was transferred) but rather next to "Tokens transferred" the data for which were extracted from the interpreted Logs
:
TL; DR
I want to be able to systematically know whether an ERC20 event includes a transfer of tokens and between which two addresses. How is this possible for custom events (i.e. events other than the ERC2O standard Approval
and Transfer
events) using web3.py
?
Approve
and theTransfer
events in order to be ERC20-compatible. What if aLend
event exists that transfers tokens as well? Is it against the standard to transfer tokens with any event other thanTransfer
? Thank you for your time