RLP Rationale Question

Question

I am trying to understand why RLP was chosen and developed as an in-house protocol.I stumbled up this link which provides some insights:

https://github.com/ethereum/wiki/wiki/Design-Rationale

Trying to understand the paragraph below, a detailed explanation is greatly appreciated.

"The alternative to RLP would have been using an existing algorithm such as protobuf or BSON; however, we prefer RLP because of (1) simplicity of implementation, and (2) guaranteed absolute byte-perfect consistency. Key/value maps in many languages don't have an explicit ordering, and floating point formats have many special cases, potentially leading to the same data leading to different encodings and thus different hashes."

Answer 1

It says that RLP is

• simple
• unambiguous

It's simpler than other algorithms because it doesn't define any data types other than bytes and arrays.

It's unambiguous because the same input data is always serialized to the same sequence of bytes.

In many languages key/value maps don't have explicit ordering. This means 2 entries:

key1: value1
key2: value2

might be serialized as:

{
  "key1": "value1",
  "key2": "value2"
}

in one implementation, and

{
  "key2": "value2",
  "key1": "value1"
}

in another implementation. The same input data can output different serializations. This is unacceptable in Ethereum.

On the other hand, when using RLP, the map needs to be converted to arrays first (this conversion is out of scope for RLP), and then passed to RLP for serialization. In our example, the input structure for RLP will be:

[["key1", "value1"],["key2","value2"]]

where strings in quotes should first be converted into bytes (also out of scope for RLP).

Basically, RLP tries to avoid ambiguity by limiting the supported datatypes.

Answer 2

RLP is an encoding scheme developed by Ethereum developers. It is a specially developed encoding scheme that is used in Ethereum to serialize binary data for storage or transmission over the network and also to save the state in a Patricia tree on storage media. It is a deterministic and consistent binary encoding scheme used to serialize objects on the Ethereum blockchain, such as account state, transactions, messages, and blocks. It operates on strings and lists to produce raw bytes that are suitable for storage and transmission. RLP is a minimalistic and simple-to-implement serialization format that does not define any data types and simply stores structures as nested arrays. In other words, RLP does not encode specific data types; instead, its primary purpose is to encode structures.

Why do we need a new encoding scheme when there are so many different serialization formats already available? The answer to this question is that RLP is a deterministic scheme, whereas other schemes may produce different results for the same input, which is absolutely unacceptable on a blockchain. Even a small change will lead to a totally different hash and will result in data integrity problems that will render the entire blockchain useless.