What is the best practise to storage large data in smart contract

Question

Contract storage is a key of 32 bytes and a value of 32 bytes. If I understand it correctly, it can support storing 2^32 bytes which is 4GB?

I have a contract code like below. It uses a map mapping(address => uint256) balances; to store users' balance. My concern is:

• If there are millions of users in the system, will it exceed the maximum storage limit? Because the map will have millions of keys, one for each user's address.

• Store a large data in the contract is very expensive. It probably costs a very high gas fee when sending a transaction.

what is the best way to solve this issue?

// SPDX-License-Identifier: MIT
// Tells the Solidity compiler to compile only from v0.8.13 to v0.9.0
pragma solidity ^0.8.13;

import "./ConvertLib.sol";

// This is just a simple example of a coin-like contract.
// It is not ERC20 compatible and cannot be expected to talk to other
// coin/token contracts.

contract MetaCoin {
    mapping(address => uint256) balances;

    uint256 transactionCount = 0;

    event Transfer(address indexed _from, address indexed _to, uint256 _value);

    constructor() {
        balances[tx.origin] = 10000;
    }

    function sendCoin(address receiver, uint256 amount)
        public
        returns (bool sufficient)
    {
        if (balances[msg.sender] < amount) return false;
        balances[msg.sender] -= amount;
        balances[receiver] += amount;
        transactionCount++;
        emit Transfer(msg.sender, receiver, amount);
        return true;
    }

    function getBalanceInEth(address addr) public view returns (uint256) {
        return ConvertLib.convert(getBalance(addr), 2);
    }

    function getBalance(address addr) public view returns (uint256) {
        return balances[addr];
    }
}

Answer 1

If there are millions of users in the system, will it exceed the maximum storage limit?

It is very unlikely to reach the storage limit with just millions of users since there are a total of 2^256 keys of storage. You will need about 1.158 x 10^77 users to reach the limits of storage.

This is a good reference for Limits of Storage in a Contract.

Store a large data in the contract is very expensive. It probably costs a very high gas fee when sending a transaction.

In the MetaCoin contract you shared above, the balances state variable is only updating 2 keys at a time when the sendCoin transaction is called. These operations are not prohibitively expensive in terms of gas fee.

The storage updates on balances state variable are happening at:

balances[msg.sender] -= amount;
balances[receiver] += amount;

Answer 2

Even when you have a lot of data in any database, the queries will take longer, the performance will decrease and overall app latency will be high. To overcome this, sharding is used. Sharding means dividing the data horizontally, based on rows, and storing each data division (division logic may be based on range or based on the key) in different shards or machines.

In the smart contract, you can implement the same strategy. If your user's amount reaches let's say 500, you can deploy a new contract. (and maybe you might proxy the requests to the newer contract based on your smart contract logic)

Or if you can somehow create a reliable oracle service to store the users off the chain and fetch the data on chain using oracle service

Answer 3

Contract storage is a key of 32 bytes and a value of 32 bytes. If I understand it correctly, it can support storing 2^32 bytes which is 4GB?

You seem to be confusing "bit" and "byte" here. A contract can store up to 2^256 values (the keyspace is 32 bytes, or 256 bits), each of 32 bytes, don't worry about running out of storage, it will never happen.

Store a large data in the contract is very expensive. It probably costs a very high gas fee when sending a transaction.

Well, that would be true if you were trying to store a large amount of data at once, which isnt the case here, any transaction you make won't update more than a few values in your contracts storage

Answer 4

It would be prohibitively expensive to store that large amount of data using blockchain technology here. I suggest you use a centralized database instead of blockchain technology. Centralized databases are fast in transaction processing, can handle large amount of data, and keeping each storage cell persistent does not require any kind of fee.

See: How much data can I store in a smart contract, what is the cost and how it is implemented?