Rinkeby is not supported by Parity because Rinkeby uses Clique consensus mechanism and Parity has not implemented this mechanism.
Kovan is not supported by Geth because Kovan uses Aura consensus mechanism (authority round) and Geth has not implemented this mechanism.
you can read more about this here.
Hope it will help.
I Personally used the 2nd option enumerated by KanekiDev.
When you start the console the first time the chain configuration is loaded from the genesis.json file, the best way is set a lot of eth in the alloc section:
I compiled a list of mobile wallets that make it possible to change the network to a testnet (Goerli, Kovan, Rinkeby or Ropsten):
MetaMask Mobile (you currently cannot connect to localhost because they only allow https dapps)
You can have a small private network of 2 or 3 nodes, for example using docker-compose.
There are a lots of tutorials out these explaining how to make sure that your nodes are connecting to each others, by passing the addresses explicitly.
If you don't like Docker, here is an example of how I setup a private net of 2 geth nodes using a Makefile: https://...