Costs/efficiency of locating stored data on Ethereum

Question

Based on the answer to this question, storing IPFS hashes on Ethereum could be done in 3 ways:

1. Store it as a string

2. Store it as a struct

3. Store it in event log

Ordered from most expensive to least expensive.

Now, assuming I would like to continue to store multiple hashes from time [t0 - t1] using one specified method, what would be the costs to locate these hashes over the blockchain beyond time t1?

For example, consider a scenario where students upload IPFS hashes of their homework assignments to the blockchain while the actual data lives on IPFS. After the submission deadline, I would like to trace who has submitted within that time frame.

Can someone compare all three methods in terms of finding these hashes?

Answer 1

TL;DR;

Events are way cheaper than contract state. Bytes32 is probably the best choice, with or without structs. Event-only is possible for a narrow band of use-cases.

Store it as a string

This is wasteful because strings are variable length and therefore require two words, one for the length and one for the payload. Since IPFS hashes are always 32 bytes, this overhead both in the interface and the storage is unnecessary.

Store it as a struct

A struct is a container, within which you can store other defined types, in effect creating a compound type. As worded, this option is invalid.

Store it in event log

Event storage is much cheaper than state storage, with the caveat that values stored in an event are inaccessible to contracts. That may be acceptable in this case.

Not mentioned

bytes32 is a fixed sized type that maps nicely to IPFS (without the leading Qm) as well as to Ethereum's natural word size and smallest addressable storage slot. This makes it seem rather ideal for IPFS hashes, whether stored in contract states or in event logs.

Realistically, you will probably end up creating a struct with metadata about the docs.

struct Doc {
  bytes32 IPFShash;
  // other stuff
}

Even the most rudimentary logic in the contract (is this a known document?) will require some state storage. Organization usually calls for a little more. So generally, while every effort should be made to keep it to a minimum, some contract storage is likely required.

And, if following best practice, also emit an event for important state changes. An event log and emitters would look something like:

event LogNewDoc(address sender, bytes hash, ...);

and later, when something happens:

emit LogNewDoc(msg.sender, ipfsHash, ...);

There is a stateless pattern that might work in the case that the contract will never inspect hashes and you want to client-side to fetch relevant events quickly.

You maintain a uint in the contract that points to the most recent block that contains an interesting event. Within the event, another pointer the event before that. That means a client doesn't need to explore the whole chain to retreive events and events arrive most-recent first. The contract state requires overwriting a uint which is currently 5,000 because it wouldn't be 0 for very long.

Hope it helps.

Elaboration

Super-simple example. A client inspects latestEvent() and then listens to that block only to fetch it then closes the listener. The breadcrumb in the log suggests where to look for an earlier event.

pragma solidity 0.5.1;

contract EventChain {

    // one of these for each chain you want conveniently accessible from client software
    uint public latestEvent;

    event LogChainedEvent(address sender, uint previousEvent);

    function logData() public {
        emit LogChainedEvent(msg.sender, latestEvent);
        latestEvent++;
    }
}

With that approach, it's conceivable to organize event chains in a way that makes data access convenient. For example, you could track a version history (IPFS hash changes every time) based on some sort of identifier for the subject matter.

Consider:

mapping(bytes32 => uint) public latestVersion; // topic => events (ipfsHash)

Answer 2

The IPFS hash [SHA2-256] will fit nicely into one bytes32, which is the smallest amount of storage in the Ethereum VM. It will require an encoder and/or a decoder, depending on your use case.

I store the hash as two hexadecimal words (instead of one binary) as the hex encoder will cost more than the savings made on storage. If you store more than one CID you may want to reconsider of course.

// The 32-byte SHA2-256 from IPFS is split over two hexadecimal words.
// Use all lower-case for valid IPFS URI composition.
bytes32 immutable FSHashHex1;
bytes32 immutable FSHashHex2;

// FSURIFix packs both the prefix and the suffix of an IPFS URI with hex
// encoding. This is to effectively store the entire URI in three words.
//
// The CID header consists of the following (multiformat) prefixes:
// 'f': lower-case hexadecimal (for all what follows)
// '01': CID version 1 (in hexadecimal)
// '70': MerkleDAG ProtoBuf (in hexadecimal)
// '12': SHA2-256 (in hexadecimal)
// '20': bit length (in hexadecimal)
bytes21 constant FSURIFix = "ipfs://f01701220.json";

function tokenURI(uint256 tokenID) override(ERC721Metadata) public view returns (string memory) {
        requireToken(tokenID);

        uint decimalPath = 0x2f303030;               // "/000"
        decimalPath += tokenID % 10;                 // digit
        decimalPath += ((tokenID / 10) % 10) << 8;   // deci digit
        decimalPath += ((tokenID / 100) % 10) << 16; // centi digit

        return string(bytes.concat(bytes16(FSURIFix), // trim head from fix
                FSHashHex1, FSHashHex2,               // both hex parts
                bytes4(uint32(decimalPath)),          // convert to bytes
                bytes5(uint40(uint168(FSURIFix))))    // trim tail from fix
        );
}