While I cannot point to a specific instance, I do know that, like you mentioned, there have been suspicious activities on gambling sites that may point to this type of behavior. Note: the below example is not about blockhashes as the source of randomness, but has the same idea.
The best example I have is the gambling site, Etheroll. There have been a few instances were players were winning much more than they should from a statistical standpoint, given their chosen chance of winning. With that said, these occurrences were in a relatively short timespan, so the explanation for the winnings may simply be due to variance.
At one point, someone was winning with such bad odds that the creator paused the game to investigate. If I recall correctly, there was nothing malicious found, but that does not mean the player was simply being lucky. The issue with this specific game is that the random number is generated from an oracle that reads from random.org, so there are potentially many layers where malicious behavior may have occurred (random.org employee, oracle employee, or miner).
Are there numbers on what the payout would need to be relative to percentage of hash-power the miner controls?
For a miner to profit from doing this, they would simply need to make more than the average block payout, which is approximately 2-2.1 ETH, at this time. Given the types of games that exist and their associated rewards, it is completely reasonable for a miner to forgo their mining reward en lieu of a payout on one of these sites.