Ethereum network messaging

Question

I am trying to "join the dots" around the networking aspect of Ethereum. From the documents, it seems, the underlying protocol is RLPx that powers the ÐΞVp2p Wire Protocol.

Though TCP provides a connection-oriented medium, ÐΞVp2p nodes communicate in terms of packets. RLPx provides facilities to send and receive packets.

There is a tangential remark in the RLPx documents that it uses RLP.

Packets are dynamically framed, prefixed with an RLP encoded header, encrypted, and authenticated. Multiplexing is achieved via the frame header which specifies the destination protocol of a packet.

The ÐΞVp2p Wire Protocol in turn supports sub-protocols of which Ethereum Wire Protocol is one.

From ÐΞVp2p docs:

ÐΞVp2p is designed to support arbitrary sub-protocols (aka capabilities) over the basic wire protocol.

From Ethereum docs:

Peer-to-peer communications between nodes running Ethereum clients run using the underlying ÐΞVp2p Wire Protocol.

Questions:

1. Can you please explain, where/how is the Ethereum sub-protocol plugged in ÐΞVp2p protocol?
2. Can APIs of an SDK (say, Javascript) be mapped to Ethereum sub-protocol message IDs?
3. Or, is the mapping to message IDs limited to clients such as Geth?

Answer 1

I wrote this on the gitter.im channel, so here it is again anyway. Caveat is I have not looked at recent geth code for this, so there may be subtle discrepancies with actuality.

A key paragraph is this "Message IDs are assumed to be compact from ID 0x10 onwards (0x00-0x10 is reserved for ÐΞVp2p messages) and given to each shared (equal-version, equal name) sub-protocol in alphabetic order. Sub-protocols that are not shared are ignored. If multiple versions are shared of the same (equal name) sub-protocol, the numerically highest wins, others are ignored."

Basically the DevP2P messages are all identified by their type, which is the first integer.

The first 0x0F(?) are reserved for DevP2P messages, specified in the DevP2P-Wire-Protocol document.

The next message ids appear to depend on what protocols are commonly supported by the pair of peers. By commonly, I mean shared.

The sub protocols shared by the peers are established in the Hello message (0x00) And the key phrase in the docs related to that is "cap Specifies a peer capability name as a length-3 ASCII string. Current supported capabilities are eth, shh."

So let's say both peers after the Hello message only support "eth", then according to the Ethereum wire protocol doc, the next message identifier is offset + 0x00 for the Ethereum "Status" message (making it 0x10)

To your question, "Suppose, I want to introduce a new sub-protocol", I think the above answers that. You would need to make a new 3 letter identifier (like, ETH, LES, SHH) and advertise that as a capability in your DevP2P client. Clients would handshake and in the Hello message establish that they have your ??? protocol in common, and that would become part of the message id space.

So, to summarise: a "sub protocol" is a message id space identified by a 3 letter name. Peers handshake and in the Hello messages specify which sub protocols they support. For that p2p relationship, the common protocols are arranged in alphabetical order, and when messages are sent, their identifier is the offset relative to the start of the message id space given to the alphabetically ordered message id space.