Appendix D of the Yellow Paper, in defining the node types, states (italics mine):
Leaf: A two-item structure whose first item corresponds to the nibbles in the key not already accounted for by the accumulation of keys and branches traversed from the root. The hex-prefix encoding method is used and the second parameter to the function is required to be true.
Appendix C defines Hex Prefix Encoding HP:
...The low nibble of the first byte is zero in the case of an even number of nibbles and the first nibble in the case of an odd number. All remaining nibbles (now an even number) fit properly into the remaining bytes
When applied to the Merkle Tree, if the key length is an odd number of nibbles, the first nibble of the key is stored in the second nibble of the prefix, else the second nibble of the prefix is set to 0. If you look closely at the drawing you'll see tiny little arrows intended to depict this. Either way (even or odd key length) the total number of nibbles is always even, which means the size of the merkle tree will always be a whole number of bytes.
Another way of putting this might be:
Given a key k with some number of nibbles, the prefix byte will be:
prefix byte key byte(s)
[nibble0, nibble1]---> Node type [stored nibbles]
[0, 0] ----------> Extension Node, n (=length(k)) is even [k[0]...k[n-1]]
[1, k[0]] ----------> Extension Node, n (=length(k)) is odd [k[1]...k[n-1]]
[2, 0] ----------> Leaf Node, n (=length(k)) is even [k[0]...k[n-1]]
[3, k[0]] ----------> Leaf Node, n (=length(k)) is odd [k[1]...k[n-1]]