How to get data location for string > 31 length

Question

The documentation says:

bytes and string are encoded identically. For byte arrays that store data which is 32 or more bytes long, the main slot stores length * 2 + 1 and the data is stored as usual in keccak256(slot).

I am working with a string (> 31 bytes), a state variable in storage, and am trying to get the location of the data. I am passing the slot number (converted to a bytes variable) to keccak256. It returns a bytes32. How do I use that to find out where the data is stored?

If it matters, I am getting the bytes32 value from this code.

bytes32 data_slot = keccak256 (uint_to_bytes (slot_number));

And the function is defined like this.

function uint_to_bytes (uint value) internal pure returns (bytes memory)
{
    uint byte_count = value <= 255 ? 1 : 2;
    bytes memory slot_contents = new bytes (byte_count);

    uint content_index = 0;
    bool value_started = false;
    for (int16 b = 31; b >= 0; b--)
    {
        uint a = uint (b);
        uint shift = a * 8;
        uint anded_value = (0xff << shift) & value;
        uint8 this_byte = uint8 (anded_value >> shift);

        if (this_byte > 0)
            value_started = true;

        if (value_started)
            slot_contents [content_index++] = byte (this_byte);
    }

    return slot_contents;
}

In my case, the base slot number is 5, which contains the length of my string. The length in that slot is correct.

But how do I get the location of the data when keccak256 returns a bytes32. When I convert the bytes32 to a uint256, I get 99383055861825221844433915613614375939768875017829574557572350922731547121557. Certainly that can't be the slot number where the data is.

Answer 1

The storage in Solidity is a mapping of bytes32 to bytes32 and a slot under the hood is represented by bytes32. I did not fully understand the code you provided but here are my two cents.

When you declare a byte[] in storage of the contract, it is packed. Which means the data against one index in that array would occupy one byte rather than full slot of 32 bytes as is the case for simple arrays. For the ordinary arrays with no packing involved the data at index 0 and 1 can be located at keccak256(slot number) and keccak256(slot number) + 1 but that is not case for byte arrays (or any packed array such as uint8[]) as keccak256(slot number) would return the data at first 32 indexes of the array because of the packing.

An example:

pragma solidity 0.5.0;

contract TestStorage {
    byte[] b;

    constructor () public {
        uint8 i;
        for (i = 1 ; i <= 32; i++) {
            b.push(byte(i));
        }
    
        for (i = 1 ; i <= 32; i++) {
            b.push(byte(i));
        }
    }

    function readFromStorage() public view returns(bytes32 slot1, bytes32 slot2, bytes32 slot3) {
        bytes memory slot = new bytes(32);
        assembly {
            mstore(
                add(slot, 32),
                b_slot
            )
        
            let slotHash := keccak256(add(slot, 32), 32)
            slot1 := sload(
                add(
                    slotHash,
                    0
                )
            )
        
            slot2 := sload(
                add(
                    slotHash,
                    1
                )
            )
        
            slot3 := sload(
                add(
                    slotHash,
                    2
                )
            )
        
        }

    }
}

Here in the constructor we populate the first 32 bytes and second 32 bytes with the same content and read in the function using assembly. Only the first two slots are populated and the third slot would be empty.

Hope it helps!