What happens to my old horse when I get a new one?
Solution 1:
They go back to where you originally got them from. The only exception to this I think is Frost who will go to Merryfair Farm. Taking him from this location will not be considered theft or alter the guards.