I'm trying to make a Dapp using solidity. I saw a few article to segregate data from codes for updating code contracts.

But what can I do if I have to upgrade data structure itself, such as adding additional fields and so on? I think the update will make a new contract and previous data will be gone.

The only way I think of is adding additional functions to retrieve all data of the data contract for future use.

Is there any other way to solve data migration needs?

Thanks in advance.


As indicated in my answer to this question, I'm a big believer in total data discovery. If you have that property, then you're at least half way to being able to migrate that data to a new structure. But it's not always possible.

You're not going to get away without some logic. Even in modular storage contracts, you're still needing permission and setter logic. Getter logic too if you're dealing with complex types.

So first strategy is simple data types to begin with and simple permissions such as a contract owner.

The simplest extensible database could be a mapping of bytes.

mapping (bytes32 => bytes) public simpleDB;

function set(bytes32 key, bytes data)
    simpleDB[key] = data;

Bytes is chosen here because any data structure can be rendered into and interpreted from an array of bytes.

Byte streams to and from contracts themselves use RLP encoding to render an object into a byte array. I haven't seen Solidity code to implement this, but if there's none out there, I'll give it a go when I have the chance.

The problem with this is that if other contracts want to interact with with such a store, there is no way of reading values without unacceptable overheads. This is not so much a problem for a front end but it does weaken the ability of inter-contract applications. Also, if the data is only being used by the front-end, then perhaps it's better stored off chain in the first place in something like IPFS or Swarm.

So with that said, I don't really have an answer but I think it's a good question deserving some research.

  • I really like this idea of total data discovery, but this doesn't necessarily have to be accomplished on chain. You can scrape the blockchain for transactions on your contract and keep an ongoing 'mirror' of the data. This is what we're working on for QuickBlocks.io (pre-alpha, but working prototype). Apr 20 '17 at 14:24
  • @ThomasJayRush It is a good idea. I also thought that kind of thing to back up data. And it also can be used for data analysis and so on. Apr 21 '17 at 1:58
  • Exactly. Data backup, test data, analysis data, data useful to a website. All sorts of things. Apr 21 '17 at 5:14

Adding my agreement to o0ragman0o, totally discoverable data is a good thing. The Mapped Structs with Delete-enabled Index pattern on this page shows a way to organize discoverable data when the field structure is known in advance and is expected to remain stable. Are there well-solved and simple storage patterns for Solidity?

If the field layout isn't known in advance or you want to maintain the option of extending it with more fields in the future (keeping in mind that off-chain storage is an option), a solution could be to treat the field lists in each record as a dynamic table of key/value pairs following a similar pattern.

Here, the fields are known by simple field numbers (uint), and it's the consumer's responsibility to assign meaning. You get a list of records known by primary key, and a list of field keys within each record, the finally a value assigned to the record/field pair of keys.

It's generalized such that new logic controllers could be introduced and the original schema extended without loss of data. Supports delete at the record and record+field level.

// Each field includes a value and a pointer to the fieldKeyIndex

struct FieldStruct {
    bytes32 value;
    uint fieldKeyListPointer;

// Each record supports enumerating key/value pairs stored

struct RecordStruct {
    mapping(uint => FieldStruct) fieldStructs;
    uint[] fieldKeyList;
    uint recordListPointer;

// Each collection supports enumerating the primary keys stored

mapping(bytes32 => RecordStruct) private recordStructs;
bytes32[] private recordList;

An experimental implementation is available over here as GeneralizedCollection.sol: https://bitbucket.org/rhitchens2/soliditystoragepatterns/overview

Hope it helps.

  • Thank you for great tips. And also your articles on medium about solidity CRUD helped me a lot to understand basic patterns. Apr 21 '17 at 2:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.