Short answer: it really depends on the token, and typically varies from 25'000 to 150'000, with very rare cases going over 500'000 gas.
Recommendation: set it 200'000 if you don't have access to an estimate, it will work 99% of the time. It is a "limit" anyway, you will get back any unspent gas immediately.
If your wallet proposes an estimate, add roughly 10-20% to it.
There are several ways to get to the numbers you are looking for.
The simplest is probably to use a good software wallet and let it do an estimate.
Many plugins and apps that allow you to send transactions on the Ethereum blockchain will tell you in advance how much gas they think the transaction will cost, and that is usually a good starting point to choose your gas limit.
What they do is actually pre-run the transaction locally, and by simulating the transaction, you get the estimated gas cost, which is often exactly correct. It may however vary, if the smart contract was called and its state changed in between the estimate and the actual mining of the tx. If the state change in some way affects your transaction, then the real gas cost may be different.
So I usually recommend to take the estimate and roughly round it up by about 10-20%. E.g. 129'543 ==> 150000. Anyway any excess gas sent along the transaction will be refunded immediately after the transaction is mined.
You can use any good wallet app or the Metamask plugin, there's many tools that could do the job.
As for which other operations make the limit rise: all of them.
Sending a transaction costs a minimum of 21000gas in principle. If the transaction involves any kind of computation or storage, then the gas cost will go up, and the gas limit you need to use must at least match it.
So any call to a smart contract function will cost 21000 plus the cost of executing its code.
Most typical arithmetic operations cost between 3 and 10gas, as do most logical, memory and flow operations. Reading a 256bits word from storage costs 50, and writing a new variable to storage costs 20000, updating it costs 5000. Getting the eth balance costs 20. Validating the address of an elliptic curve signature costs 3000.
So a typical erc20 transfer will cost around 45k gas if the recipient didnt have any tokens before: 21k base + 20k to store the brand new token balance for this user + a few thousand for the validations, flow and event. A second transfer to the same rexipient will cost closer to 30, since his balance is now non-zero.
This should get you an idea of what's lying under the hood, and you can refer to the yellow paper for detailed gas costs of all opcodes.