Is the network ID truly unique? It's hard to believe this, as it's just an integer.
A geth node must have the same networkid as well as the same genesis block in order to join a network. The geth node also needs to know the enodeid of the peers you want to connect to. The geth client has a hardcoded list of bootnodes for the mainnet and official testnets.
You'll simply be connected to a different network if you have the same networkid but the genesis block is different.