How do I make my own horse follow me?

I've found a horse in Skryim and I ride it a bit to explore the wild but when I dismount, it wanders away and doesn't stay where I leave it. How do I stop it from doing this? Is there a special thing I forgot about horses? I didn't steal this horse - it was from an Imperial Fort that I've cleared.

You will need to actually buy a horse for 1000G or earn one through a quest, wild or stolen horses will usually end up back where they started. Once you own a horse they will not necessarily follow behind you though either.

The horse you own will always wait for you outside towns etc. at the local stables. They will also be waiting somewhere nearby when you spawn after a fast-travel. So if you lose your horse in the wild the quickest and easiest way to get him back is to fast-travel somewhere (if the horse is still alive).

Only a horse that you own will follow you. This means either purchasing one from a stable for 1000g or earning one via a quest reward such as the dark brotherhood's Shadowmare.

Wild and stolen horses don't follow you.

If you are on the PC version, you can open the console, click on the horse, and type "setownership".