I would like to understand exactly which signatures are used in the Ethereum. Here https://bitcoin.stackexchange.com/questions/12554/why-the-signature-is-always-65-13232-bytes-long it says that there are DER encoded and custom encoded (also called recoverable) signatures in bitcoin. Can I use both types of these signatures in Ethereum? Are there any types besides these two? Is it possible to convert one type to another?
Ethereum uses RLP encoding of signatures and Bitcoin uses DER encoding. You cannot use DER encoded signatures in Ethereum, only RLP encoded signatures. The underlying numbers (R and S) are the same in that they are derived from the same elliptic curve (called secp256k1).
An Ethereum signature contains three values: v, R, S. A Bitcoin signature contains only two numbers: R and S. The extra value that Ethereum uses makes it possible to recover the public key from the signature which means that an Ethereum transaction does not include the public key. Bitcoin does not use this trick, thus a signed Bitcoin transaction is larger than a signed Ethereum transaction, because it also includes the public key.
It is possible to convert an RLP signature to a DER encoded signature but not the other way around since the DER signature does not contain the v value. The v value can for practical purposes only have two values (representing the parity of a number).