the abi specification of solidity which provides functions like abi.encodePacked() using a certain type of encoding (eg: for functions, a 4 byte function signature hash followed by encoding of parameters). My question is, is there a name for such a particular encoding format? Are we using RLP encoding? If not, why can't RLP type encoding be used for abi specification? From what I understand RLP is more "self describing" and might possibly not require an abi file as a descriptor.
May be I don’t fully understand your question, but I can say that it should be taken into account that ABI is not the internal coding for the contract, but the description of the interface in a human readable format, from which it is possible to build the internal jump function call hash table, which is ordered and used as pure RLP.
In particular the order of the entries is that specified for [ki, vi] in RLP definition. [[k1, v1], [k2, v2], ...] are sorted using the standard ordering for strings, as prescribed.
In the bytecode there is not stored any ABI, but the cited function call hash table. Knowing the ABI is needed outside the contract, in order to have them defined and calculable the ‘ki’ cited above, where the ‘vi’ are the addresses where to jump to in order to call the ‘i’ function.
On the other side, the “abi encoding” that is realized by the solidity function you cited is simply a pattern of zero padding for each element and composition of them in a single structure, nothing more, as described here: https://link.medium.com/zVpLc3qihT
I hope to have added some information which helps.