How can I keep the Lord of the Mountain as a mount?

Unfortunately, you can't keep it as a permanent mount, but you are able to ride it, as you've noticed.

According to this guide on taming the creature, while you are able to tame it, you can't register it to a stable.

Sadly, you can’t register it at a stable, since it’s not [technically] a horse, but it’s a beautiful creature and it would be a shame if you missed it.

According to users in this thread, the monster saddle won't allow you to register, it either.

Have you obtained the monster saddle? Someone was theorizing that he, or some other creature you can't normally register, could be registered with the use of the monster set, but I haven't been able to [test] it.

It doesn't work.