Is there anyway to set (white-list?) your Ethereum node's peers by public key and ip address? I'm specifically playing around with a private network right now, but this should be relevant to the public network as well.
Yes, you can make a file called static-nodes.json, containing the enode address (basically IP + public key).
Alternatively I haven't tried this but according to this answer you can make a file called trusted-nodes.json with the same content, then set max peers to a low number (zero?) so your node doesn't connect to anyone else.
To be on the safe side you may well want to firewall your nodes off as well, either by specifying their IPs or by connecting them via SSH tunnels or a VPN or whatever.