1

I have a contract that has a struct including address and unit, and mapping of a struct, like this.

  struct User {
    uint count;
    address userAddress;
  }
  mapping (address => User) users;

There is a point at which I need to make a copy of this mapping and clear out the entries of the old one, like this pseudo code:

  users_backup = users;
  users = [];

Of course this code won't work, so I learned you can loop through it like this:

Assignment of mapping in Solidity

But that seems expensive, especially after reading:

Copying a mapping from Contract A to contract B

So How do I do this efficiently?

With a mapping of 100k users or 1 million or even 100 million entries would this looping method be feasible? Or would it be better to change my architecture?

uint[] userCounts;
address[] userAddresses;

I could use a series of arrays instead of maps. Then, would I be able to copy one array to a new variable and clear the old variable out in one or two lines with less gas? The Hard thing is, even the arrays will have to be variable arrays because I can't know beforehand how many users will use it.

What would you do to minimize gas consumption? I'm just not sure what the best pattern is, thanks!

  • I'm sure someone can work it out, but we have to find a new approach to "There is a point at which I need to make a copy of this mapping." There is no good way to do a large-scale reorganization. What do we need to accomplish? – Rob Hitchens Jul 5 at 1:38
  • @RobHitchens that's a good point. I have thought about an alternative approach that is probably the right way, but I'm not quite sure how to do it. I could possibly solve the problem by instantiating a new version of the contract since I'm essentially trying to restore most of the state back to its original 'constructor' position, (with the added additional users_backup variable). but I'm new to solidity so that seemed more complex than just resetting all the variables in the same contract... – Legit Stack Jul 5 at 4:17
  • Well, the patterns are quite different. Why do you think you need to do any of that? Can you describe a requirement without falling back on implementation details? The method of accomplishing something efficiency is likely to be quite different from inituitions about how to start. – Rob Hitchens Jul 5 at 4:23
  • @RobHitchens the requirements without implementation details are simply that the contract is cyclical: it cycles on a yearly basis such that in order to do the calculations during the next year just some of the previous year's data is required, but also we must generate/record new data during the current year. This cyclical requirement is why I thought of copying one variable over another, and consequently produced the mapping loop solution I presented above. simply put is there a better way? – Legit Stack Jul 5 at 5:32
  • There is a point at which I need to make a copy of this mapping and clear out the entries of the old one - now, why would you ever wanna do something like that? I recommend that you review your design and find out how you got to this point to begin with. I can't think of any reason whatsoever, of having to replicate a mapping and then clear the old one. – goodvibration Jul 5 at 6:11
1

It's not easy to conjure up a trivial example. Based on the comments, I think the main concerns are approximately:

  1. Clean up expired information responsibly and capture the gas refund offered for releasing unneeded storage.
  2. Avoid iteration. https://blog.b9lab.com/getting-loopy-with-solidity-1d51794622ad
  3. Access information in O(1), by user, by epoch, globally by epoch, or globally overall.

You may have more paths to consider. You have to anticipate all queries the contract needs to make against its own data and work out a way to access the answer with O(1) complexity.

This approach is to give you some ideas:

  1. You'll need data organization that easily contends with using id sets that refer to mappings of structs. It usually works best to have a global mapping of everything that exists, with lists of pointers that satisfy each case the contract needs for its own lookup logic. For example, a global list of all transaction IDs that exist and similar lists for each user, in each epoch. You can use address(this) as a "User" to represent the global set. You will write keys in multiple places but the records only once.
  2. You will want to add another dimension to some parts of the structure to logically group things into epochs (e.g. a year). That which should be logically cleared out goes in the Epoch layer and initializes to zero, naturally. That which should not reset every year goes in the layer above so that it appears to carry forward. To be clear, it is imperative that the data stays at rest because large-scale reorganization is not feasible or desirable.
  3. If you are done with the data you should be a good citizen and do garbage collection. This will also help you economize on gas cost for using the system, because each time a user deletes a non-zero value, they get a gas refund. You have to do that without iteration. A solution to that problem is to use the structure that exists to inspect the expired Epoch. If data still exists, delete one record. You can do that every time a user supplies gas for any purpose by attaching a modifier to state-changing functions. This should not objectionable because users get a gas refund if there is something to delete.

You can make life a lot easier with these Set libraries that attend to CRUD operations. https://github.com/rob-Hitchens/SetTypes. In case the method being used isn't clear, check out this series: https://medium.com/robhitchens/solidity-crud-epilogue-e563e794fde

You can do all sorts of data handling as libraries - linked lists, sorted lists, etc.. For something as complex as your projects sounds, I recommend separating those concerns from the line-of-business application to reduce repetition and improve readability and probably reliability.

The garbage collection method suggested is based on the notion of work amortization explained over here: https://medium.com/@weka/dividend-bearing-tokens-on-ethereum-42d01c710657. The current Epoch and the proper functioning of the contract is the primary concern, of course. It should be irrelevant if obsolete data is cleaned up, but, it is good policy to do so, and you can make it so users are rewarded for chipping away at it until the cleanup process is complete.

Roughly:

modifier garbageCollection {
  if(previousEpochStillExists) {
    // get to work removing one txn/record and all references to it
  }
  if(nothingLeft)
    // remove the epoch itsef
  }
  _;
}

As an after-thought, think carefully about minimizing state storage. You can immortalize inputs with event logs (way cheaper). A good heuristic is to limit state storage to values the contract needs for its own internal logic. Never for memre record-keeping because a client might be interested. There are more efficient ways to do that.

Hope it helps.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.