Can I set the gas price to what ever I want?
Yes, you can. But miners have a default strategy for determining gas price to charge and if the amount you're willing to pay is below that, your transaction will be rejected. (Try setting the slider all the way to the left in Mist and try to send a transaction.)
What are these limits actually?
Completely arbitrary and made up by the gas price oracle running on miners. There is no necessity to use this oracle, you could swap it out for a really greedy or really kind one. It probably helps the stability of the network to leave things as is though.
What are realistic gas prices, what happens if it is chosen too low or
Realistic would be assuming most people don't mess with the gas price oracle settings when they mine and they use all defaults. Which means as you fill up more and more of the 3,141,592 maximum gas in the block, you'll need to provide a higher and higher gas price. (Which by default discourages overusing gas / filling blocks.) Too low = your transaction won't go through. Too high = miners will probably thank you. (There is no too high with gas prices.)