Taken from the documentation, you have the following axioms:
if a string is 0-55 bytes long, the RLP encoding consists of a single byte with value 0x80 plus the length of the string followed by the string. The range of the first byte is thus [0x80, 0xb7].
And
If the total payload of a list (i.e. the combined length of all its items being RLP encoded) is 0-55 bytes long, the RLP encoding consists of a single byte with value 0xc0 plus the length of the list followed by the concatenation of the RLP encodings of the items. The range of the first byte is thus [0xc0, 0xf7].
Hence, ["cat",[[]],"pig",[""],"sheep"]["cat",[[]],"pig",[""],"sheep"] would be encoded with the following logic:
You would encore cat like 0x83, 'c', 'a', 't'
"cat" is a string, hence you start from 0x80 and add the length of the string (3) so it gives us 0x83, then the characters of the string.
[[]] would be 0xc1, 0xc0
This array contains ONE empty array.
As 0xc1 is the length of the array (the array contain one element, here [] ), and 0xc0 is the length of the second array (empty element)
"pig" is same as "cat", a string of 3 characters so:
0x83, 'p', 'i', 'g'
[""] is an array of ONE element being an empty string (0 characters) so it would encode as 0xc1, 0x80 (0xc0 + 1 , 0x80 + 0)
"sheep" is a 5 characters string, so it would encode as:
0x85, 's', 'h', 'e', 'e', 'p' (0x80 + 5, and each letter of the string)
The whole array contains 18 bytes of value, so it would encode as:
[ 0xd2, 0x83, 'c', 'a', 't', 0xc1, 0xc0, 0x83, 'p', 'i', 'g', 0xc1, 0x80, 0x85, 's', 'h', 'e', 'e', 'p'] (0xd2 is 0xc0 + 0x12, 0x12 being 18 in decimal)
For clarity purpose, characters have not been encoded in hexadecimal, but the real encoding will look like this:
[0xd2, 0x83, 0x63, 0x61, 0x74, 0xc1, 0xc0, 0x83, 0x70, 0x69, 0x67, 0xc1, 0x80, 0x85, 0x73, 0x68, 0x65, 0x65, 0x70]