There are many proposals of adding precompiled contracts for different reasons. I do not, though, understand why a precompiled contract would be more efficient than an actual contract. If that is the case, doesn't that mean that the EVM isn't sufficiently optimized? What is stopping Ethereum of compiling every contract; perhaps using LLVM?
A precompiled contract, as I understand it, is actually protocol-subsidized. That is, while there is technically code at a given address, the gas price is far less than if it was run manually. If I'm understanding the yellow paper correctly, ECDSARECOVER costs only 3000 gas.
This works because more efficient external code can be written for various implementations (if it doesn't exist already), and then just plugged in when the contract is run. But this isn't necessarily the case for some random contract off the blockchain.
I believe geth has a limited JIT compiler for smart contracts, which leads into another issue with compiling every contract: space. There's only so much room to reasonably set aside for caching compiled contracts, especially considering potential attackers. geth currently appears to limit it to 64 contracts by default, and I've heard significant discussion on gitter on algorithms (not necessarily geth-related) to pick which ones.
Matthew's answer is correct. However, I will add that one of the goals of Ethereum 2.0 is to get rid of those pre-compiled contracts. Since they must be implemented by all clients, the risk of running into segmentation issues is high. This risk is mitigated by keeping the amount of pre-compiled contracts extremely small (4 at the moment; another 4 in the coming January update IIRC).
Pre-compiled contracts can be avoided if on-chain contracts implement them or call into on-chain library contracts that implement them. I believe this is one of the many design goals of moving away from legacy EVM code to eWASM, which would have a sufficiently fast VM to support this feature.