7

I am using Etherscan API for various Ethereum details.

I need to find out the tokens transferred from a token original contract (let's say A) address to a particular address(let's say B).

For this I would need to iterate through all transactions of B and all the Event Logs of A and see if any transaction in Events matches that of B if yes then the data key gives me the value of token transfer.

But I have no way to know these transactions would be part of which block so I would have to iterate through all the blocks, since API supports max of 1000 results in a single call I have no way to know at how many blocks this limit would be reached, so I iterate at steps of 1000, it is painstakingly slow and would take hours.

The API accepts fromBlock and toBlock parameter. Presently the latest block is at ~4287262 How can I effectively paginate the results, given if I use a step of 1000 for blocks it would take ~4000 calls which seems very ineffective.

One way I optimized is seeing at which block the contract was created this effectively eliminates 95% of the blocks and used multithreading for other blocks but it is still slow.

Any other way/method/suggestion would be great.

Improve this question
3
  • why do you think this is inefficient? as it has to do 4000 calls ? but anyway your idea is to paginate you don't need the results at once? Is it time wise effectiveness or a data wise effectiveness you want?
    – Achala Dissanayake
    Commented Sep 18, 2017 at 19:32
  • 1
    I want all the results should be fetched in just a few milliseconds like a single API call, even with my optimizations and multithreading it takes few seconds. Otherwise it would take hours.
    – garg10may
    Commented Sep 19, 2017 at 7:38
  • anybody knows where the number/limit 1000 appears in the documentation ? couldn't find anything on web3.js
    – Adil
    Commented Mar 22, 2018 at 18:34

1 Answer 1

Reset to default
0

I assume what you meant by ineffective is time wise or data wise.

Since your idea is to paginate the results you will not need all the API calls to happen at once.,As you can call only for the section which is need to be viewed at the moment, hence in time wise the API design to fetch only 1000 results is not going to be a problem (Trying to get all the results may cause even a higher delay at initial rendering - and I think fetching only 1000 results is actually better for a paginated view).

And your consideration about efficiency is in data wise(amount of data transferred), you are going to save only the data used API call request and response headers. Compared to the amount of data transferred header sizes are negligible. I checked it with Postman request to the etherscan event log API and results were as follow,

  • Without data (only headers): response size = 310 B enter image description here

  • With data : response size = 210.5 KB enter image description here

So the amount of data you are trying to save is very less compared to data transferred and if not all the logs are viewed, having all of them loaded early will actually be a waste of data. Hence the API design is effective in data wise as well.

And as per my understanding, since this is the API providers design, if you still want to do it, option is to find another API to get event data.

You may refer links here and here as well.

Hope this helps.

Improve this answer
6
  • 1
    I explained the problem in detail, the API is very limited I need like all the results at once so that can find the tokens. Don't know how does they display the data on their website.
    – garg10may
    Commented Sep 19, 2017 at 7:49
  • my answer was on your idea of paginating results. Anyhow do you need all the results to be displayed at once? (if it's to get a count or something like that you'll need all the results at once. If it's to display you can set the view section by section and as explained above it's effective)
    – Achala Dissanayake
    Commented Sep 19, 2017 at 8:23
  • 1
    I don't need to display the data, I just need to find out value of token transfer at a address for a particular transaction. But this seems infeasible, so If I can find the total value of tokens at that address, it would also do. so looking for some API for that. Query is simple. How many tokens( a particular ERC20 token) does this address contain.
    – garg10may
    Commented Sep 19, 2017 at 8:33
  • If it's the case that depends on the api design
    – Achala Dissanayake
    Commented Sep 19, 2017 at 8:49
  • What I can suggest is you can search for more apis available. If you couldn't find a api to match your requirement, you may keep a background process to fetch data from the etherscan.io and store it in your local storage( where you host your application), this may need to be done periodically, and you can design that storage to query efficiently the results for your requirement. Or you may check to develop an API yourself via web3
    – Achala Dissanayake
    Commented Sep 19, 2017 at 8:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.