11

If I store file chunks in the blocks, I'm causing duplicated blocks all over the networks (I mean, my file's chunks will be in each computer on the blockchain network, So eventually I have many many copies of this file in each block of each computer). On the other hand, If I store reference to the file in the block(for example, the block will hold: Myfile.txt stores in IP: 123.123.123.123 and 11.11.11.11) each computer that has my file, can change it easily. and even If i'm checking whether 123.123.123.123 or 11.11.11.11 (my example above) changed my file, and the answer is "True" (by comparing file's hash before and after storage) , I still don't get my file, and the data storage on blockchain is pointless.

So, how does companies like Storj works? how do they store the data on the blockchain? (As I understood, storj store references on the blocks, to 3 computers that holds my file, so how do they avoid my problem which I mentioned above?)

Thanks in advance.

Improve this question

2 Answers 2

Reset to default
11

Short answer is don't store chunks of files in the blockchain. It's not well-suited for this purpose.

Better answer. It helps to consider separate concerns:

  1. Smart Contracts. Minimalist data that must be true for all users at all times. Focus is on fidelity, as it's extremely difficult to post false data into a contract that's careful about updates.
  2. Distributed Storage of large objects. Swarm, Storj and IPFS are examples of distributed object storage/distributed file system. They use various strategies for smearing out copies and shards across participating nodes, but they are not putting all things in all places at all times.

At a very high level, we can consider a combination of these approaches. A distributed storage system can provide resilient storage of a blob, media, or JSON object with a lot of data, known by it's file name or path; a unique identifer.

A Smart Contract can track the identifier of the object that is valid, now. It can hold a validation hash useful for confirming that the correct data has been loaded from the "other" source.

For example, one might have a contract that says "There is a movie called HomeMovie" located at "url ..." and its hash is "0x456" and this is the "valid" object as of this time. If an authorized user changes the file (version), they would update the Smart Contract; "Now, the movie we're using is HomeMovie2 located at "new url ..." and its hash is "0x567".

This keeps the Ethereum storage to a minimum, while providing a way to confirm no corruption of the data in the overall system. There is an extensive assortment of projects working toward a unified view of things; that is, one interface that does it all.

Hope it helps.

Update: See Blog: Simple Storage Patterns in Solidity

Improve this answer
3
  • Thanks for the answer. Your answer lead me to another question. If we stick to your example of the smart contract of the "HomeMovie" - if the user that stores this movie, change this file and corrupted it, I can recognize it by the fact that the hash before isn't equal to the current hash (after the change). But now what? companies like Storj can't just say to the client who store the movie, that his file corrupted and he can't get his HomeMovie anymore.. what is the point here? how companies like Storj handle the fact that any user can change the files they store?
    – Roy Kuper
    Feb 9, 2017 at 16:09
  • 2
    The distributed storage systems all use mechanisms to ensure integrity. My point was just to highlight that Smart Contracts aren't always the best place to store everything. Especially the bulky stuff. Another scenario has large database records stored "somewhere" and a smart contract that knows the correct hash of each record's current data. Now everyone else can verify the authenticity of the record details using a free read-only function in a contract. Things move along with minimal storage footprint in the Ethereum blockchain. .
    – Rob Hitchens
    Feb 9, 2017 at 17:03
  • Saw this and thought of you: ethereum.stackexchange.com/questions/12068/…
    – Rob Hitchens
    Feb 10, 2017 at 20:04
2

After reading the white paper, I understood that the storage of shards is peer to peer or in Storj terms between owner and farmers only. They use blockchain technology to implement smart contracts between owner and farmer which is needed as service level agreement is required in rented storage. Another place where they've used block Chain is to pay the farmers using storjcoin/bitcoins. Both of these applications are well full-filled by blockChain.

I did not see anywhere in paper mentioning that pointer or data block being stored on all the nodes in network. Please point out the link if I misunderstood this.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.