If done right, it can actually save gas.
When you free up storage, you receive a gas refund. If you delete the order in the same transaction that completes it, your completion tx will actually end up costing less!
For instance, if the completion process costs 1000 gas, and you receive a refund of 200 for freeing up storage, the tx cost at the end will be 800.
Of course, this doesn't make as much sense if you are making another tx especially to delete the order, since the storage refund will not be more than the intrinsic gas fee, and will end up costing more.
Costs aside, cleaning up storage you no longer need is just being a good citizen of the chain and preventing unnecessary bloat.