How to programmatically detect and accept ETH and ERC20 deposits

Question

I'm looking to replicate the functionality provided by cryptocurrency exchanges such as Kraken and Poloniex when it comes to depositing ETH and tokens. Namely:

• Users can generate and send funds to one or more deposit addresses for ETH, EOS, OMG, and other tokens
• The relevant user's balance is updated when a deposit is detected for a given token or asset
• Sent funds are stored in accounts controlled by the exchange

What's the best way to achieve this?

---

I have a few ideas using web3, but I'm not sure they're the most efficient or scalable solutions:

ETH

• For each user who wants to make a deposit, generate an ETH address and private key using web3.eth.accounts.create(). Map the generated address to the user and store the private key.
• Use web3.eth.filter to monitor the latest block on the ETH blockchain. Match transactions in the block against the user addresses stored and update user balances as appropriate.

Issues with this approach: monitoring each block and matching against stored addresses is computationally intensive especially if the number of addresses tracked becomes large (hundreds of thousands). How can this scale?

ERC20 Tokens

• For each user who wants to make a deposit, generate an ETH address and private key using web3.eth.accounts.create(). Map the generated address to the user and relevant token, and store the private key.
• ERC20 contracts emit Transfer events when a transaction happens on the contract. Monitor this event using web3.eth.filter. When a Transfer event occurs for an address that exists in our database, update the relevant user's balance on the exchange with the transferred amount.

Issues with this approach: How do you set up the web3 filter to detect Transfer events only from the token in question, not the entire network? Should I set up 1 filter per address, or 1 filter for all user addresses? How many addresses can web3 filter track at a time? What if the tracked addresses become very high (hundreds of thousands)?

---

Is this the right way to think about all this or am I missing something? I'm surprised there isn't more obvious documentation around this flow since it's quite popular (in exchanges and any other service that accepts digital currency payments).

Answer 1

It'd probably go something like this:

1. User wants to deposit a token to your exchange

   1. If it's the first time the user is depositing that token then this occurs:

      1. Generate a new wallet key/pair on the server side
      2. Encrypt the private key with a secret/salt only known by the server and store the encrypted private key and public address in your database, along with the info about the token they generated it for and the user id who initiated it

   2. Next time they hit the deposit button to get the deposit address you'll already have a unique deposit address for the user stored in the database that you can present to the user

2. The user deposits token to the deposit address

   1. On your server have a worker that listens to all of the ethereum blockchain events. Check the transaction's "to" address and check it against the deposit addresses you have stored. When there's a match, send a notification to the user regarding the deposit event

3. Call the token contracts balanceOf method passing it the deposit address you've generated tied to the user in order to display the users token balance in the exchange. You only display that particular token balance and nothing else for that address.

4. The user wants to withdraw their token

   1. Your exchange will have a liquidity pool on stand by. You'll have to transfer some ETH from your liquidity pool to the user's deposit address in order to pay for the gas cost. You include this cost as part of the withdrawal fee. You already have the encrypted private stored so you decrypt it with the secret/salt, sign the transfer transaction and broadcast it to the ethereum network

Answer 2

I have the same thoughts about implementation. And I think that your proposed solution is good. But for both, ether and ERC20 tokens, I do not recommend monitoring the blockchain.

For ether balance you will call web3.eth.getBalance passing the address to check the value when the user enter the page and put a javascript timer to perform the call every 15 seconds for example.

And, the same thing, regarding your question for Ether balance the ERC20 tokens' balances: "How do you set up the web3 filter to detect Transfer events only from the token in question, not the entire network?". The best way I think is to call the smart contract of the ERC20 token to inquire the balance of each token. This is by calling the method: function balanceOf(address _owner) for every token once the user enter the page. And you can set a javascript timer to refresh the values every 15 seconds for example.

I think that this is what Kraken do for example since the page contents is reloaded ever couple of seconds.

Note that you will do ajax call to refresh the values of the balances without the need to refresh all the page. Additionally, calling balanceOf is free since it is read-only method that will be executed at your directly-connected node without any gas cost.

Answer 3

You need a HD (Hierarchical Determenistic) wallet. Check out Truffle's HD wallet npm package. This link might help: https://www.npmjs.com/package/@truffle/hdwallet-provider

Answer 4

First of all, better use HD wallet, so that you dont need to generate random addresses, instead just save mnemonic in a .env file and ... You can do it using Ethers.js :

const hdNode = ethers.utils.HDNode.fromMnemonic(mnemonic);
const newUser = hdNode.derivePath(`m/44'/60'/0'/0/${uid}`);

Also keep in mind that if you are using Infura or other node providers, you probably have a rate limit. So you can not check thousands of addresses the way you mentioned expecialy if you have thousands of users.

What can you do is to use Ethers.js again, and in intervals you get transactions info of blocks like this:

    const provider = new ethers.providers.InfuraProvider("goerli" , APIKEY)

    for (let i = lastCheckedBlock + 1; i <= latestBlock; i++) {
         let block = await provider.getBlockWithTransactions( i )
         console.log(block.transactions)
        }

Keep in mind that you gonna wait for the tx to be confirmed if you want to make sure the deposit is really OK, if its for an Crypto Exchange project.