Dealing with unbounded loops

Question

Knowing that we shouldn't loop through arrays that can potentially be so big that the function iterating over them would run out of gas, would the following work?

If I had an array with a million or more records (addresses) and I wanted to extract a list of all its contents. Would I be able to do it by using a limit + offset?

The function would iterate over the big array from array[offset] to array[offset+limit] and return an array with those elements.

Would this work even if the original array I'm iterating over was huge? (as long as limit is a relatively small number).

Answer 1

It's not a terrible idea and it reminds me of a linked list project I played around with, in which a "hint" was passed in to help reduce search iterations.

I came to realize that like many other considerations, sort order was something that can usually be dealt with off-chain.

Although you can't be sure who/what will call your function, it might help to consider the expected usage.

If it's an external client or server, you might find it works better to emit events with each insertion/deletion and there may be no need to pass such large chunks of information in one response.

If it's expected to be another contract, it's probably best to deal with it one row at a time for the best economy.

You may find it works best to maintain an array for a list, and a mapping for O(1) lookups based on the address keys. You want all ops to be O(1).

Hope it helps.