The difficulty is automatically adjusted based on block intervals. If the interval gets too short, the difficulty level is raised and it gets harder to find the right solution for the mining puzzle. And vice-versa.
Scaling isn't actually related to the difficulty too much. It's mostly about all the data in the network which needs to be processed by all nodes. Also all nodes need to propagate all the data around and reach consensus. The more there are nodes, the more difficult this all gets.
But here is a quick introduction to difficulty so you can hopefully understand more about the why part.
According to https://dltlabs.com/how-difficulty-adjustment-algorithm-works-in-ethereum/ the aim is to get block time between 10 and 19 seconds. Now what would happen if these numbers were smaller (faster blocks) or greater (slower blocks)?
Reaching consensus would become more difficult as the blocks would have less time to propagate along the network. This means more uncle blocks, more chain reorganizations and therefore a bit less security.
This would also mean that slower computers could not keep up with the synchronization process and mining could only be done on faster computers - dedicated servers for example. This would lead to centralization.
This is basically the opposite of the other option. Network would get more decentralized, reaching consensus would be easier and more nodes would join the network. Sounds good, right?
Well, this would also mean that delay between blocks is longer and therefore all real-world interactions with the blockchain would take longer. If an exchange earlier required 10 confirmations before a block is considered confirmed, reaching 10 confirmations would now take longer.
Also, if the block gas limit is not adjusted accordingly, the network could handle less transactions in any given time limit: a block has the same amount of transactions as earlier but block interval is longer.
There are a lot of aspects to consider. Above are just some of them. It's a delicate balance between speed and security/decentralization/etc.