From my reading of the Yellow Paper (YP), Yes you would be able to spend the funds of the contract. But as you note, it is with extremely low probability of being able to find a contract's private key.
Spending money in Ethereum is done via a CALL opcode, passing it parameters such as the
value (being spent). Since you have the private key, you can create a valid signature for the transaction, which the network will accept. There is no special check on the
sender apart from the validity of the signature. Below are the checks in the YP, and the others are easy to satisfy (for example, an account's transaction nonce is easily looked up).
- The transaction is well-formed RLP, with no ad- ditional trailing
- The transaction signature is valid.
- The transaction nonce is valid (equivalent to the sender account’s
- The gas limit is no smaller than the intrinsic gas, g0, used by the
- The sender account balance contains at least the cost, v0, required
in up-front payment.
EDIT June 2021: EIP-3607 has been drafted so that it would not be possible to spend a contract's funds using a private key: such transactions would be rejected as invalid.