I am learning Ethereum using Geth. I have a private blockchain and I am saving hashes through a REST API (python web service). I have readed I can save hash values using maps (mapping data type) and using logs. I am using the logs option and it works fine, but I want to know if it is the most efficient way, or if I should use a map or other kind of variable (like a string or byte array). I need to save millions of hashes (about 1 per second) so I am worried about the long-term scalability, mainly related to the search. By now I have a service saveHash(string hexHashValue) returning the transaction hash, and a service verifyHash(string hexHashValue, string transactionHash).

In python using web3.py I have this functions, I don't now if this can be improved, if using a map I can save millions of values in a better way or if I can find a specific string value in the contract using other methods. So my question is if the best way is logs, maps, strings or if the difference is trivial.

The contract:

pragma solidity ^0.5.0;

contract HashSave {                   
   event log_current_hash(bytes hashArg);

   constructor() public {                       

   function set_hash(bytes memory objectHash) public {
       emit log_current_hash(objectHash);

My current test methods using logs (event log_current_hash):

def save_new_hash(hash_bytes):
    tx_hash = contract_instance.functions.set_hash(hash_bytes).transact()   
    log_test_file.write("Hash: 0x%s Tx: %s\r\n" % (hash_bytes.hex(), tx_hash.hex()))
    return tx_hash.hex()

def check_transaction_exists(tx_hex_hash, obj_hash_bytes):
        tx_data = w3.eth.getTransaction(tx_hex_hash)
        if tx_data is not None:
            print("Transaction is in the blockchain")
            if find_obj_hash_in_transaction(tx_data.blockNumber, obj_hash_bytes) is True:
                return "Data validated ok"
                return "Data is not valid"
    except Exception:
        return "Transaction does't exists"

def find_obj_hash_in_transaction(block_num, obj_hash_bytes):
    hash_filter = contract_instance.events.log_current_hash.createFilter(fromBlock=block_num, toBlock=block_num,
                                                                       argument_filters={'hashArg': obj_hash_bytes})
    if hash_filter is None:
        return False
    hash_events = hash_filter.get_all_entries()
    if len(hash_events) > 0:
        return True
        return False
  • 1
    Current Ethereum clients are not well suited to store large amounts of data. An option is to store data off-chain in a merkle-tree like structure so you can generate a proof to validate if the data was tampered with. And store this proof in a smart contract. – Ismael Jan 15 at 17:45
  • @Ismael thanks for your response. I thought one transaction per second wasn't large amounts of data for a private blockchain (e.g. 5 PCs in a room, about 5-10 MB transactions per day), but maybe you're right. I could save daily or hourly data in some database and like you say save some hash of all the info or a merkle tree root in the blockchain. – RobertGG Jan 15 at 18:09
  • It doesn't appear to be much right now, but your client will be creating and storing a cryptographic proof every time. The other issue is that you need other tools to make useful queries to your data, like running a block explorer or something similar. – Ismael Jan 16 at 14:02
  • Ok @Ismael I understand you. Now about your suggestion of using tools, I suppose you are talking from a management perspective and not about my functions. Until now I though blocks and network explorers where mainly for admin purposes (I'm currently using eth-netstats), but my idea was to call functions (like check_transaction_exists) from a remote location (e.g. using a web service). When you say "tools to make useful queries", are you talking about management and good practices right? because I can't connect a web service to a tool, I need a custom function for that. Thanks in advance. – RobertGG Jan 19 at 3:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.