Can that be done by a simple token transfer to my contract's account? And, how?
I am assuming we are talking about regular ERC20 tokens that you issued through a crowdsale and contributors bought.
If that's so, the only way for you to buy the tokens back would be for the current token holders to transfer them back to you by using the token's transfer() method.
Somehow, you would have to get in touch with someone holding a balance in your token contract and have them send the tokens in exchange of ether. You'd probably want to do it through a custom smart contract as well to protect both parties while doing the exchange. If not, you have to trust the other person that after you send them the ether they will send you the tokens or the other way around.
The other way would be to buy them back through a public exchange platform like EtherDelta if they are being publicly traded.