Two bottleneck types
There are two types of speed limits that are in play here. These are latency and throughput. Latency is the amount of time one must wait until a transaction is processed. The other is throughput: the number of transactions that can be processed in a particular amount of time. Imagine the difference between a supermarket with 100 patrons and 10 checkout counters and a grocery store with 5 patrons and one checkout counter. Now, imagine each checkout counter can process one patron per minute. In the first case, the average person waits 5 minutes to get checked out. In the second, each person waits an average of 2.5 minutes. On the other hand, the supermarket can handle 10x as many customers per minute. Thus, a busy supermarket might have higher throughput than a small grocery store, but the latency might be higher. Now imagine that both stores are closed for 20 consecutive hours a day and gets 100 customers a day. Essentially, you need to wait, on average, 10 hours to buy your groceries (the checkout time is negligible). You can see that the checkout counters show low utilization.
In Ethereum, the minimum time between blocks is, on average, around 25 seconds at the moment. Thus, the minimum latency is an average of about 12.5 seconds (i.e., if your transaction is selected for the next block, you wait on average around 12.5 seconds for it to be included). If only one transaction were sent per minute, this would have no effect on the average latency and the throughput would be one block per minute. The CPU utilization for processing blocks would remain very low. For technical reasons, the interval between blocks was previously set to a lower value to enable the network to stay in sync (i.e., so that nodes can reach consensus). This number could technically be lower, and it used to be. The average block time is rising due to the Ethereum ice age/difficulty bomb, a non-technical reason, which is being used to provide strong incentives to developers to hurry up the introduce proof-of-stake (PoS) to replace proof of work and for nodes to upgrade their clients when the PoS mechanism is introduced. Since much of the time is spent waiting for information, CPU utilization remains low.
Meanwhile, there are technical limitations regarding throughput. The first of these limits is practicality: it would be very difficulty for many/most node operators to run a node if the storage capacity required to store the blockchain grew by 1TB a week since I'm guessing not many people run (or even know how to run) systems that enable scaling their storage at that rate, if at all. There is an economic aspect to this as well: Even if people knew how, it would be costly for someone just trying to add a node for verifying transactions to keep up with that kind of growth rate.
A second technical limit on throughput is there are limits on nodes' processing and storage speed meaning that some nodes would be unable to verify blocks quickly enough to stay in sync without some sort of limit on the processing/processing involved for the transactions in each block. This results, for the same reason, in a limit on the number of transactions mining nodes could include in a block. Without a limit on processing/storage per block, it would be harder to determine what fees to include for a transaction to ensure timely processing: the number of transactions included per block could vary drastically, depending on which miner processes a block, and the network would often be out of sync. Sharding is a way around this technical limitation. Note that while waiting for a disk to return data, the CPU is twiddling its thumbs and you'd see low CPU utilization. Even if blocks ran up against processing limits on some mining nodes, faster nodes would take less time to verify a block's transactions and wouldn't see a 100% CPU utilization rate.
A third limit arises out of the technical limits on throughput above: an artificial limit on the amount of processing/storage that can be used by transactions in a block comes in the form of the block gas limit. I assume one of the reasons why this is in place is to avoid hitting either of the two technical limits. This makes it easier and cheaper to run a node. In that situation, the technical limits on throughput rise proportionally with the number of shards and the block gas limit could be raised that way, too. In practice, it may not be desirable to increase the block gas limit directly proportionally to the technical throughput of the network: I assume it is desirable for any node to be able to verify that the Ethereum network is operating as designed (i.e., to detect cheaters); as such, a node could randomly check shards' correct operation. The chance of detecting cheaters decreases with the number of shards. If you require dedicating a node to testing a single shard -- in order to keep up with the transactions -- then you can use the birthday paradox to your advantage to maintain a sub-linear growth in number of verification nodes (relative to total number of shards) to detect cheaters with reasonably high probability.
Both latency and throughput bottlenecks exist in Ethereum and there are solutions being worked on to work around both of them. The current Ethereum bottlenecks are artificial (ice age and block gas limit) but are useful to enable relatively inexpensive Ethereum nodes to exist and provide a more level playing field for users of the Ethereum network.