I have a large data array in the BC. The array has 15,000,000 items of Car struct

struct Car{
 string tag;
 uint mileage;
 string make;
 string model;
 address address;
 string country;

I need to search for a: specific model and
millage between X , Y and
country equals to X and
tag starts with a specific letter
(it gets more complicated).

Do you think its a good practice to pass back to WEB3 15M items and search in JS?
Doing this search in solidity will cost a lot of gas. and JS will be easier to search in.

How fast will WEB3 return this amount of rows?


1 Answer 1


That sounds hairy.

Did the Car insertion emit an event with the relevant information? If yes, then you can .watch it from the block of the first insertion. This would:

  • help you reduce the memory cost in Javascript
  • allow you to "restart" from a more recent block whenever it crashed

Have a look at RxJs on how to transform an event watcher into an Observable. Then you can apply filters and such, which would help you.

Something like this not so pseudo-code:

Rx.Observable.create(observer => {
        const filter = myContract.LogCar();

        filter.watch((err, oneEvent) => {
            if (err) observer.onError(err)
            else observer.onNext(oneEvent);

        // Cleanup logic
        return () => filter.stopWatching(console.error);
    .filter(oneEvent => oneEvent.args.mileage.atLeast(10))
        () => console.log("completed"));
  • Thank you, can you please explain how an event can help with that? what happens when someone enters a new search query, all the events already fired. please explain. thanks
    – SexyMF
    Jul 7, 2018 at 16:11
  • If you want to run a search query, the blockchain itself is the wrong data structure. To run a search query, you need to import all the events into a specialised search engine. To extract all the events, you can use the same code as above, minus the filter. Jul 8, 2018 at 10:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.