Storage and memory layout of strings

Question

How exactly are strings stored in storage? And, how does that differ from the way they're stored in memory when loaded from storage into a memory variable?

Answer 1

Layout of State Variables in Storage > bytes and string

bytes and string are encoded identically. In general, the encoding is similar to bytes1[], in the sense that there is a slot for the array itself and a data area that is computed using a keccak256 hash of that slot’s position. However, for short values (shorter than 32 bytes) the array elements are stored together with the length in the same slot.

In particular: if the data is at most 31 bytes long, the elements are stored in the higher-order bytes (left aligned) and the lowest-order byte stores the value length * 2. For byte arrays that store data which is 32 or more bytes long, the main slot p stores length * 2 + 1 and the data is stored as usual in keccak256(p). This means that you can distinguish a short array from a long array by checking if the lowest bit is set: short (not set) and long (set).

So for example a string "abcd" will use the short format and take up a single slot with the upper four bytes being characters a, b, c, d and remaining bytes being zeros.

On the other hand "0123456789012345678901234567890123456789", which has more than 32 characters will take 3 slots. A single slot placed at the normal location between other storage variables will only contain the length (40) shifted left by one bit and will have the last bit set to indicate that it's the long format. I.e. its value will be 81. The other two slots will store the characters of the string (with unused space being filled with zeros) and the compiler will place them at a location computed by hashing the address of the slot containing length with keccak256().

Layout in Memory

Elements in memory arrays in Solidity always occupy multiples of 32 bytes (this is even true for bytes1[], but not for bytes and string). Multi-dimensional memory arrays are pointers to memory arrays. The length of a dynamic array is stored at the first slot of the array and followed by the array elements.

In memory "abcd" will occupy two consecutive 32-byte slots. The first will store the length and the other the characters.

"0123456789012345678901234567890123456789" will occupy 3 consecutive slots with the first containing the length (40) and the other two storing the characters. There's no short/long format in memory so the length is not shifted like it would be in storage.

Answer 2

I've been exploring how Solidity 0.8.21 manages storage for bytes and string data types and found some interesting details that may help those who are looking into the same topic:

1. Short Strings (Less than 32 bytes): If a string is shorter than 32 bytes, Solidity stores the actual string in the higher bytes of the storage slot. The lower bytes contain the length of the string, shifted left by one bit.

2. Long Strings (More than 31 bytes): If a string is longer than 31 bytes, the lower bytes of the storage slot will contain the string's length, shifted left by one bit, with an additional 1 in the least significant bit. The actual string data is stored in the storage at the offset calculated using keccak256(data.slot) and in the subsequent slots.

This is crucial for understanding how Solidity optimizes storage and might be helpful for those working on lower-level operations or optimization.