1

I'm currently working on dApps for a private blockchain based on ethereum. While it works fine and the dApps really show the power of the blockchain it lacks the speed and versatility of a SQL database. (I know i know blockchain != database).

Example where it lacks is getting specific data. Because you don't have SQL like queries you need to loop through a lot of data. Or if have a array of a struct you want that back, this also poses an issue.

So I was thinking about an caching mechanism to cache data that is verified by the network in a local database for fast querying.

Google failed when searching for this so my question does something like this really not exist? If it doesn't exists what do I need to keep in mind if I want to design a system like this myself.

Improve this question

1 Answer 1

Reset to default
3

A two-part answer.

"you need to loop": This just jumps out at me and deserves a comment. Smart Contracts don't give you indexed storage but this doesn't imply that a contract should loop over disorganized data. In fact, contracts must not loop over unbounded sets. Instead, it means that contracts are responsible for organizing data internally. Organize pointers/indexes to support random access and interation as needed. Some examples here. See Mapped Struct with Index: Are there well-solved and simple storage patterns for Solidity?

Contracts mustn't loop over rows/instances because that will tend toward a rising transaction cost. More round trips = higher cost. At the block gas limit all transactions will fail. Not good.

DB Cache: It's perfectly valid to create an off-chain database that is bound to an on-chain source. The on-chain source is usually considered authoritative. If you follow the practice that all significant state changes should be accompanied by an emitted event, then a server can listen for create, update, delete operations. When this is done diligently, it should be possible to bootstrap an empty database and recreate the entire state history of the contract - the database should catch up to the live state.

Sites like etherscan.io and exchanges use similar approaches. Visitors are really "hitting" databases, for performance reasons, and not the actual blockchain.

Hope it helps.

Improve this answer
4
  • Out of the loop was maybe the wrong thing to say as you said looping is something you really shouldn't do. The question relates more to if off chain caching is something you should do, and if so how would you tackle this / or does a tool already exists for ethereum blockchain.
    – just_trying_stuff
    Jun 22, 2017 at 10:02
  • 3
    "a server can listen for create, update, delete operations". Event emitters are a really important feature of Smart Contracts that enable parties outside the blockchain to "listen" for events or even replay past events in order. This is enabling since the off-chain database can include all sorts of indexing and optimizations.
    – Rob Hitchens
    Jun 22, 2017 at 15:48
  • Is there a generic way to get the events. As I understand correctly from the documentation you can set the event emitter only on contract level to get the data. Can you for instance set an event emitter on new block added which links to the correct contract. I know there are events on new blocks added but that doesn't give you the information that an contract event gives you, right?
    – just_trying_stuff
    Jun 27, 2017 at 13:22
  • This looks like a pretty good example: ethereum.stackexchange.com/questions/17214/…
    – Rob Hitchens
    Jun 27, 2017 at 16:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.