As complex computations cost a lot of gas (sometimes more than actually forecasted), so are there any popular ways or techniques where I can optimize a complex computation to a simplified version of the same, for saving on the gas costs?
Given the high cost of blockchain-based computation, it is definitely worthwhile to performance optimise the computations performed in the chain itself. One basic technique used by almost all developers of blockchain-based applications is to develop applications in a testing environment where the gas cost of using the application can be repeatedly measured without deploying to the public Ethereum blockchain and incurring an actual cost in terms of ether. Once the application has reduced gas needs to reasonable levels, then it can be deployed in the wild!
It's also important to use the appropriate opcodes/pseudocontracts for frequently re-used operations such as calculating of hashes or verifying of signatures. The gas costs of these methods are substantially lower than implementing the operation yourself within the scripting language of the Ethereum Virtual Machine.
There are a few techniques which can be used depending on what you're optimizing for:
Some contracts are pre-compiled, allowing for quick access in Solidity to otherwise complex functions such as ECRECOVER, RIPEMD160, SHA256.