Increasing the block gas limit entails more transactions and/or more expensive transactions can be included in the block. This has several consequences.
The first consequence of increased supply of transaction resources is that, in theory, this leads to lower prices for transactions (which may be made up for by increased volume).
More transactions (or more complex ones) also means larger blocks which translates to increased storage requirements. Miners may wish to limit the growth of the blockchain's storage requirements.
The third consequence is it takes longer for nodes to process transactions in the blocks. This makes it harder for lower-end nodes to stay in sync with the network.
Larger blocks lead to more uncles.
Mis-priced op-codes on the EVM can lead to attacks. While higher block limits do not necessarily lead to increased block sizes (a miner may opt to simply ignore all transactions), miners could also vote to lower the gas limit to minimize damage from other miners who do mine malicious transactions.
Another reason why miners may not opt to increase block gas limits is because they might be voting. At least one bit of information can be encoded by the block gas limit relative to the prior block. I'm pretty sure this has been used before, but I couldn't find a reference (I thought it was used for EIP 150 or some other fork).