7

I have a smart contract which contains some article data stored inside a struct, which is mapped with an id which uniquely identifies it with a mapping, like this:

struct Foo {
    string a;
    string b;
}

mapping (uint64 => Foo) public articles;

(Obviously the above code is simplied, but the structure is the same, although the real struct has 7 variables instead of 2).

Now I have to be able to search using any of the 7 attributes. The blockchain is going to be private, so gas usage is not a concern.

My first solution was to add a search method to the smart contract and, within it, traverse the entire mapping and check which items satisfy the condition. Obviously this is O(n) and it was taking around 8 seconds for 2000 elements, which although isn't bad, I hope for a better solution.

Then I thought about having 7 more mappings which would act as search indexes, mapping each attribute value to the index of the article that satisfies it, such as:

mapping (string => uint64) public searchIndex;

(Note that right now it isn't required to check substrings, only equality)

And finally, I was thinking about if it would be good to do keep these indexes on Elastic Search rather than on the blockchain itself, although I would prefer a pure blockchain-based solution if possible.

So, what of these options (of any other better you know) do you think is more adequate?

Thank you.

1

2 Answers 2

3

It is clear you can't directly get the object you want without knowing it's key. The idea of the 7 mappings is actually the most obvious and the least complicated one and it works in most cases not only yours.

The most useful idea I can come up with is that you look for patterns in your struct attributes, patterns that help you organize a search schema in case you have 10 or 20 thousand elements. e.g. ; if string a values are linked to a known timeline like a == "X12" for today and the next day another entry would havea == "X13". So when you want to search for structs with a == X32 you will have better understanding of where could that struct be and what are the search index limits.

Thus making searching in 20 thousand elements a search in only 2 thousand.

hope it was clear.

2

One thing I have done to solve this same type of problem is to mirror the data coming into the contract in a Redis or Mongo database to create a much more queryable dataset. Then, whenever I query that database, I verify the data in the blockchain using a getter function to ensure it hasn't been tampered with. I realize this is not the purest blockchain-only method, but it works without risking gas-heavy loops or multiple maps that multiply storage cost for what should be simple queries on non-key attributes.

Given your simplified contract:

struct Foo {
    string a;
    string b;
}

mapping (uint64 => Foo) public articles;

You can create an event to emit whenever a new addition is created like this that returns the values and the unique ID:

event newArticleCreated(string a, string b, index uint64);

Then, using the Ethers.js library (or another like Web3) you can set up a listener to catch any instances of this event and handle the data however you wish from there. In this case, you would pass the data to a service that adds the data to Mongo or Redis with the key being the uint64 key in your contract:

    //import the ethers.js library
    const ethers = require('ethers');

    //set your provider and create a new instance of your contract by providing 
    //the ABI and deployed address 
    let provider = new ethers.providers.JsonRpcProvider(<RPC URL for your node>); 
    contractInstance = new ethers.Contract(<contractAddress>, <abi>, provider);

    //listen for events matching this name
    contractInstance.on("newArticleCreated", (a, b, index) => {
        //do stuff when the event fires

        //set the values in the event to a JSON object structure
        let chainObj = {
            id: index, 
            a: a
            b: b
        }

        //add the data to the database
        dataService.addDataToDatabase(chainObj);

I have personally had success with this type of setup and it has a lot of flexibility. It allows you to query data mirrored from the chain, but always verify that data back with the contract using that index/unique ID.

Thanks!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.