An Ethereum address of an Externally Owned Acccount (EOA) does not differ in the format from an Contract Account (CA). You can lookup how the address of a CA is created in this (clickme) great post. It is not impossible to find a collision. In my opinion, it is even highly propable that collisions between EOA and CA accounts do exist. I am not sure though if an EOA with the same address as a CA can transfer the founds, I assume it is possible though.
Both an EOA and a CA are stored as an Account data structure having a nonce and a balance in the world state of ethereum. A CA also has a CodeHash (to find its code in the state database) and a StorageRoot (to find its data in the state database). When you call a smart contract from an EOA using a transaction, you supply the contract address and an entry point (function identifier) in the data section of the transaction. It can be resolved by looking up the CodeHash entry in the Account data structure. This way the Ethereum Virtual Machine (EVM) knows that you invoke a smart contract and where it can find the referenced code. My guess is, that if you send a transaction from an EOA that has the same address as a CA, you will simply transfer the funds because it is a valid transaction and the EOA and CA share the same data structure for a balance and a nonce within the Account entry in the world state. But at the same time, you can send a transaction that invokes the smart contract by including appropriate information in the data section of the transaction. This is just a guess based on my understanding, there might be some other protection mechanisms I don't know about and I appreciate if somebody can clarify that.
Another way to argue about this problem: If somebody creates a new ethereum EOA and the address does collide with the address of a CA (highly improbable), would it be fair to block the EOA from withdrawing funds? The person owning the EOA would probably not know what is going on and just pass the address to someone to transfer funds to it. To block that EOA from executing transfers would not be fair at all in my opinion.