23

Following Q/A (What datatype should I use for an IPFS address hash?) recommend us to use bytes to store IPFS hash.

I was using following example(https://github.com/AdrianClv/ethereum-ipfs/blob/master/NotSoSimpleStorage.sol) which uses string to store IPFS hash that costs around 110,000 gas price, which seem pretty expensive.

[Q] Does using bytes instead of string in order to store IPFS hash would cost cheaper? I observe that storing bytes instead of string costs very close to string (110,000 gas). Since both datatypes storage seems expensive should I use events to store them?

Is there any example/tutorial related to store IPFS hash using bytes?

Would this work:

myContract.insertHash("QmWmyoMoctfbAaiEs2G46gpeUmhqFRDW6KWo64y5r581Vz");

contract Example_bytes {
    bytes[] list;
    function insertHash(bytes ipfsHash) {
       list.push(ipfsHash); //costs around 110,000 gas. 
    }
}

contract Example_string {
    struct hashes{
         string hash;
    }

    hashes[] list;
    function insertHash(string ipfsHash) {
       list.push(hashes{hash: ipfsHash); //costs around 110,000 gas. 
    }
}
35

Your example shows storing an IPFS identity using it's alphanumeric encoding (Qm...), which is the same Base58 encoding that Bitcoin uses. However, what it's representing at its core is a number (the hash). Storing the identifier in the Base58 format needs to be a String because it includes letters (and what actually gets saved is the ASCII code for each alphanumeric character in the identifier). That means you need 46 bytes to store QmWmyoMoctfbAaiEs2G46gpeUmhqFRDW6KWo64y5r581Vz from your example.

However, that identifier can also be expressed in hexadecimal as 12207D5A99F603F231D53A4F39D1521F98D2E8BB279CF29BEBFD0687DC98458E7F89, which is only 34 bytes long (takes 68 characters to write out in hexadecimal, since every two characters in hex is a byte of data).

But, both of those are greater than 32 bytes, which is the max fixed-size byte array, so they're going to need to use a dynamically-sized byte array to store (bytes or string, both of which are expensive, as you noted).

BUT, that IPFS hash is actually two concatenated pieces. It's a multihash identifier, so the first two bytes indicate the hash function being used and the size. 0x12 is sha2, 0x20 is 256-bits long. Currently, that's the only format IPFS uses, so you could just chop off the first two bytes, which leaves you with a 32-byte value, which is small enough to fit in a bytes32 fixed-size byte array, and you save some space there (and when retrieving either your contract can re-attach 0x1220 to the front of it, or your clients need to be smart enough to do that after retrieving the value).

If you want to make sure your code is future-proof, though, you probably want to save that hash function code and size, which you could combine with the hash as a struct:

struct Multihash {
  bytes32 hash
  uint8 hash_function
  uint8 size
}

That will work with any multihash format, as long as size is less than or equal to 32 (any bigger and the actual payload won't fit in the hash property). This struct will take two storage slots (two 32-byte chunks) to store, since the two uint8 pieces can be put in one slot. You could also add up to 30 bytes of additional data to this struct without taking any more storage cost.

  • Sorry I did not get how you obtained: 12207D5A99F603F231D53A4F39D1521F98D2E8BB279CF29BEBFD0687DC98458E7F89. Here(codebeautify.org/string-hex-converter) when I convert QmWmyoMoctfbAaiEs2G46gpeUmhqFRDW6KWo64y5r581Vz into hex I got much larger string as 516d576d796f4d6f63746662416169457332473436677065556d687146524457364b576f3634793572353831567a. @MidnightLightning – alper Jun 2 '17 at 20:35
  • 2
    The conversion you did (string-hex-converter) is taking the string "QmWmy..." and showing you how the string would be stored (the ASCII value for "Q" is 0x51, "m" is 0x6d, etc.). What I did is use a tool that does Base58-decoding and used that to get the actual number being represented by that Base58 string. – MidnightLightning Jun 2 '17 at 20:55
  • 1
    uint8 function is storing an unsigned integer value as the name "function" in that struct object. "Function" is probably not the best name for that since it's a special word in Solidity and other programming languages; I picked it because in the multihash standard, that's what they call that variable; the variable that tells you what hashing function was used for this particular record (e.g. 0x12 for "sha2"). I'll update my answer to not use that special word "function" to be more clear. – MidnightLightning Jun 2 '17 at 20:58
  • 1
    Thank you so much for this. You really saved me a lot of time and effort here. This is exactly what I needed. – rhlsthrm Feb 15 '18 at 0:17
  • 2
    Thank you for saving me a ton of time. I put your approach in an end-to-end example and hope it saves some time for other people: github.com/saurfang/ipfs-multihash-on-solidity – Saurfang Feb 19 '18 at 0:01
5

Here are some js functions for stripping and re-adding the first two bytes containing the hash function and size, suitable for web3.

import bs58 from 'bs58'

// Return bytes32 hex string from base58 encoded ipfs hash,
// stripping leading 2 bytes from 34 byte IPFS hash
// Assume IPFS defaults: function:0x12=sha2, size:0x20=256 bits
// E.g. "QmNSUYVKDSvPUnRLKmuxk9diJ6yS96r1TrAXzjTiBcCLAL" -->
// "0x017dfd85d4f6cb4dcd715a88101f7b1f06cd1e009b2327a0809d01eb9c91f231"

getBytes32FromIpfsHash(ipfsListing) {
  return "0x"+bs58.decode(ipfsListing).slice(2).toString('hex')
}

// Return base58 encoded ipfs hash from bytes32 hex string,
// E.g. "0x017dfd85d4f6cb4dcd715a88101f7b1f06cd1e009b2327a0809d01eb9c91f231"
// --> "QmNSUYVKDSvPUnRLKmuxk9diJ6yS96r1TrAXzjTiBcCLAL"

getIpfsHashFromBytes32(bytes32Hex) {
  // Add our default ipfs values for first 2 bytes:
  // function:0x12=sha2, size:0x20=256 bits
  // and cut off leading "0x"
  const hashHex = "1220" + bytes32Hex.slice(2)
  const hashBytes = Buffer.from(hashHex, 'hex');
  const hashStr = bs58.encode(hashBytes)
  return hashStr
}

Here is the function used in context, calling a Listing contract deployed with Truffle.

submitListing(ipfsListing, ethPrice, units) {
  return new Promise((resolve, reject) => {
    this.listingContract.setProvider(window.web3.currentProvider)
    window.web3.eth.getAccounts((error, accounts) => {
      this.listingContract.deployed().then((instance) => {
        let weiToGive = window.web3.toWei(ethPrice, 'ether')
        return instance.create(
          this.getBytes32FromIpfsHash(ipfsListing), /*** IPFS here ***/
          weiToGive,
          units,
          {from: accounts[0]})
      }).then((result) => {
        resolve(result)
      }).catch((error) => {
        console.error("Error submitting to the Ethereum blockchain: " + error)
        reject(error)
      })
    })
  })

Taken from my work on the Origin Demo Dapp here: https://github.com/OriginProtocol/origin-js/blob/1cfc84d4693974bbf18e345e6c0def843321130c/src/services/contract-service.js#L102-L128

3

I have handled similiar situation with this util function in web3.py:

import base58

def convertIpfsBytes32(hash_string):           
  bytes_array = base58.b58decode(hash_string) 
  return bytes_array[2:]

You need base58 module. Concept is same as accepted answer.

0

Here's a more complete example using the js-multihash library:

MyContract.sol

pragma solidity ^0.4.24;

contract MyContract {

  event AddFile(address indexed owner, bytes32 digest, bytes2 hashFunction, uint8 size, bytes4 storageEngine);

  function addFile(bytes32 _digest, bytes2 _hashFunction, uint8 _size, bytes4 _storageEnginge) public {
    emit AddFile(msg.sender, _digest, _hashFunction, _size, _storageEngine);
  }
}

Javascript

import Web3 from 'web3'
import multihashes from 'multihashes'
import ipfsAPI from 'ipfs-api'

var web3 = window.web3
web3 = new Web3(web3.currentProvider)

// Utility functions:
const utils = {
  ipfs2multihash (hash) {
    let mh = multihashes.fromB58String(Buffer.from(hash))
    return {
      hashFunction: '0x' + mh.slice(0, 2).toString('hex'),
      digest: '0x' + mh.slice(2).toString('hex'),
      size: mh.length - 2
    }
  },

  multihash2hash (hashFunction, digest, size, storageEngine) {
    storageEngine = web3.toAscii(storageEngine)

    if (storageEngine === 'ipfs') {
      hashFunction = hashFunction.substr(2)
      digest = digest.substr(2)
      return {
        hash: multihashes.toB58String(multihashes.fromHexString(hashFunction + digest)),
        engine: storageEngine
      }
    }

    throw new Error('Unknown storage engine:', storageEngine)
  }
}

// ... code to instantiate contract
// ... code to get the file buffer

ipfs.add(buffer)
  .then((response) => {
    console.log('ipfs hash:', response[0].hash)

    // Prepare data
    let mh = utils.ipfs2multihash(response[0].hash)
    let storageEnginge = web3.fromAscii('ipfs')

    // Call contract
    myContractInstance.addFile.sendTransaction(mh.digest, mh.hashFunction, mh.size, storageEnginge, {from: myAccount, gas: 1000000}, (error, result) => {
      if (error) throw error
      console.log(result)
    })
  })

Reading the data

let fileFilter = myContract.AddFile({
    owner: myAccount
  }, {
       fromBlock: 0,
       toBlock: 'latest'
     }).watch((error, log) => {
      if (error) reject(error)

      console.log('file log:', log)

      let hash = utils.multihash2hash(log.args.hashFunction, log.args.digest, log.args.size, log.args.storageEngine)
      console.log('Hash:', hash)

      fileFilter.stopWatching()
    })

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.