Difference between a byte in a bytes32 and a byte in a bytes structure

Question

One can read in the docs:

A bytes is similar to byte[], but it is packed tightly in calldata and memory.

What does this exactly mean? For instance, according to this link, why should we convert a bytes object to a bytes32 one using:

function bytesToBytes32(bytes b, uint offset) private pure returns (bytes32) {
  bytes32 out;

  for (uint i = 0; i < 32; i++) {
    out |= bytes32(b[offset + i] & 0xFF) >> (i * 8);
  }
  return out;
}

and not using:

function bytesToBytes32(bytes b, uint offset) private pure returns (bytes32) {
  bytes32 out;

  for (uint i = 0; i < 32; i++) {
    out[i] = b[offset + i];
  }
  return out;
}

Furthermore:

1. Why isn't it possible to set an element of a bytes32 structure individually, but it is possible with bytes?
2. If I want to do some computations with bytes (litterally bytes, not the type bytes) by considering them as uint8, will it work for both structures by simply casting b[i] to uint8(b[i])? For instance, in order to get an uint256 from these types, would the following code:

uint res;

for (uint i = 0; i < 32; i++) {
    res += uint8(b[i]) * (256 ** i);
}

work, be b be a bytes object or a bytes32 one? Finally, are uint res = uint(b[i]); and uint res = uint(uint8(b[i])); two equivalent codes?

Answer 1

The basic unit of work of the EVM, usually called a word, is 32 bytes. This means that for the EVM, no matter whether we want to express 1 or 2^150, it will take 32 bytes, or 256 bits, to do so.

Everything costs gas in the EVM, including memory, so the fewer words the better.

A bytes is similar to byte[], but it is packed tightly in calldata and memory What does this exactly mean?

Here, being packed tightly means that rather than using one word (32 bytes) per item, the compiler will "pack" multiple items to share a single word.

Let's look at an example. Say we want to express the number 12345 as bytes, which gives [48, 57] in big endian (48 * 16^2 + 57 = 12345). If we do not pack the bytes, this will give us: [48, 0,...31 times, 57, 0,...31 times] Now, if we pack the bytes, we can express this as: [48, 57, 0,...30 times]

So as you can see, using a packed representation reduces the number of words we need to use. It is especially true with bytes, given that we waste 31 bytes per word without packing.

Why isn't it possible to set an element of a bytes32 structure individually, but it is possible with bytes?

I think this is more a design choice of the Solidity developers than anything else. I am not entirely sure about the rationale behind it but I can imagine that bytes32 was mostly designed to hold short strings and making them immutable allows for some optimizations which could be tedious otherwise.

With this in mind, the code snippet using out[i] = b[offset + i] would work in theory but is simply not valid Solidity code.

The first snippet works but is, I think, a bit awkward. b[offset + i] is a single byte, so masking it with 0xff is a no-op. The line could simply be written

out |= bytes32(b[offset + i]) >> (i * 8);

I am pretty sure that would be semantically equivalent but please correct me if I missed something.

Note that this conversion is not really related to the bytes being packed or not.

If I want to do some computations with bytes (literally bytes, not the type bytes) by considering them as uint8, will it work for both structures by simply casting b[i] to uint8(b[i])

Yes, both uint8 and a byte represent 8 bits of data and have the same range, so this should work without any problem.

Finally, are uint res = uint(b[i]); and uint res = uint(uint8(b[i])); two equivalent codes?

In theory both should be equivalent but the Solidity compiler, at least in the current version, does not allow to convert a byte into a uint directly so the explicit conversion to uint8 is necessary.