2

Storing string is expensive on smart-contracts and charge us expensive fees even if we want to store them in Events. Instead I want to store each string's decoded version in a smart-contract in order to save some memory and gas-fee charge. Overall, when I return string's decoded version, it also should be able to decoded into the original string.

As an example I want to pass my string as compressing base64 input format and its code. But when I stored compressed base64 string on a smart-contract, its returned version is not same as the one I passed.

Example example.js:

var Base64String = require('./base64-string');

var string = "QmWmyoMoctfbAaiEs2G46gpeUmhqFRDW6KWo64y5r581Vm";
console.log("Size of sample is: " + string.length);
var compressed = Base64String.compress(string);
console.log("compressed:" + compressed)
console.log("Size of compressed sample is: " + compressed.length);
string = Base64String.decompress(compressed);
console.log("Sample is: " + string);

Results node example.js: //as you can see 46 length string is compressed into 19 length.

Size of sample is: 46
compressed: ł斦쪃⡲ퟛƨ蒳憸幒桪ᔐ훨ꖨ릯鼵嗐
Size of compressed sample is: 19
Sample is: QmWmyoMoctfbAaiEs2G46gpeUmhqFRDW6KWo64y5r581VmAA

Passed base64 compressed string: //I want to store it on my smart-contract.

ł斦쪃⡲ퟛƨ蒳憸幒桪ᔐ훨ꖨ릯鼵嗐

Contract.set("ł斦쪃⡲ퟛƨ蒳憸幒桪ᔐ훨ꖨ릯鼵嗐");

From the smart-contract its returned version seems not same as the string I passed. // Contract.get()

ł斦쪃⡲<d7db>ƨ蒳憸幒桪ᔐ훨<a5a8>릯鼵嗐

Overall: "ł斦쪃⡲ퟛƨ蒳憸幒桪ᔐ훨ꖨ릯鼵嗐" != "ł斦쪃⡲<d7db>ƨ蒳憸幒桪ᔐ훨<a5a8>릯鼵嗐"

Please note that I have tried this on remix-solidity.

-

pragma solidity ^0.4.0;
contract Contract {
    string str;

    function set(string val) public {
        str = val;
    }

    function get() public constant returns (string) {
        return str;
    }
}

Please also not that we can provide string's IPFS-hash, but still I want to compress the IPFS-hash which will bring the string size from 46 to 19, this will save nearly around more than half memory.

So possible approach would be:

Large File → IPFS-hash (46 characters) → compress base64(IPFS-hash) (19 characters)

[Q] How should I pass compressed base64 string into a smart-contract? or basically what should I do to store my string compressed version on a smart-contract that when I returned it, it should match with the string I passed?

Thank you for your valuable time and help.

1

An IPFS address consists, in reverse order, of a hash of the data, the length of the said hash, and an identifier to tell you what algorithm is being used for the hashing. The algorithms in wide use produce hashes 32 bytes long. This is then encoded into a base58 string.

To store this efficiently you should undo the base58 encoding to get bytes. You shouldn't need to store the data length since it is implied by the version, so you need 32 bytes, plus a way to know the version. 32 bytes fits exactly into an EVM storage slot.

The 32 byte hash is effectively random, so you shouldn't expect to be able to compress it any smaller, and even if you could it may not help you because often you'd still end up using a 32-byte storage slot. If storage is very tight, you may choose to ignore the version identifier and just require that everybody use sha2-256, which is the current de-facto standard. If you need future-proofing, you may need an additional variable for this.

1

Have you thought about using a peer-to-peer distributed file system to store the data? For example, IPFS. This means that all you need to store in the smart contract is the strings 'content hash' i.e. Multihash and Base58 encoding.

Therefore, strings of any length can be digested into a fixed length hash; minimising the amount of bytes you need to store on chain.

Check out another answer I gave that further explains how this works here.

For the example you have provided: It looks like your output is different from the string that you initially compress.

Inputted:

var string = "QmWmyoMoctfbAaiEs2G46gpeUmhqFRDW6KWo64y5r581Vm";

Your Output from Base64String.decompress(compressed):

Sample is: QmWmyoMoctfbAaiEs2G46gpeUmhqFRDW6KWo64y5r581VmAA

Note the extra "AA" appended to the end of the string. Looks like a potential problem with the library in use.

EDIT: Instead of further compressing the IPFS hash to try and save around 50% on memory. You could use one IPFS hash that references a file containing multiple strings. You're DApp would know the format of this file so it'll be able to interpret it correctly. For example a CSV file or "string1_string2_string3". Potential to save >50% on memory this way.

Hope this helps, feel free to leave me a comment for any clarification. Thanks :)

  • That's possible, on the example the string I provide is actually an IPFS-hash similiar to your NotSoSimpleStorage.sol contract. So still I want to compress the IPFS-hash which will bring the string size from 46 to 19, this will save nearly around half memory. @Malone. – alper Oct 15 '17 at 11:03
  • You could add another level of indirection. What I mean is, index your strings in the smart contract, for example index 0 will reference an IPFS hash where the mapping is done outside of the smart contract and in your DApp. This may incur some higher level design decisions. One other more simple suggestion, is store the IPFS hash to a file containing more than one string. Your application will know the file format and know how to read consecutive strings in the file. I'm pretty sure you'll be able to save >50% memory this way. – Malone Oct 15 '17 at 11:17
  • What do you mean by "You could add another level of indirection."? I approve that addressing could done with IPFS-hash, but we should also store the IPFS-hash inside an Event where some-other user wants to access the data could access the IPFS-hash through event and can use it as a key. Other-wise only provider of the ipfs-hash could access the data since he is the only one who knows the key. @Malone. – alper Oct 15 '17 at 11:22
  • 1
    Thank you nice approach to store more than one string having a separator in between "string1_string2_string3" and store it under ipfs. @Malone – alper Oct 15 '17 at 11:23
  • 1
    Yes, even better. It really depends on how you want to store the data. E.g. you could be XML and use XPath to traverse the structure. Though JSON is would be best for JS application. – Malone Oct 15 '17 at 11:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.