The Ethereum Virtual Machine (EVM) is the computer that all full nodes in the Ethereum network agree to run. When there is code / data on the blockchain, consensus is needed to agree on what that code does. Everyone agrees on how the EVM should behave, and everyone has the same data on the blockchain, so everyone will compute the same answers. From this perspective, it looks like a single World Computer leading to that description.
Not all nodes perform the code execution. There are different types of nodes and light clients, such as those on mobile or devices with less resources, do not perform all the code execution. However, this means that light clients need to place trust on some service and slightly more vulnerable to getting deceived.
All full nodes perform the code execution for their own security. By doing this, full nodes can verify the computation result on their own, and they do not have reliance on some service. So instead of why is the miner the only node that gets the block reward, it's more about why do other nodes decide to perform the computation (or not).