How do I completely remove an old version of Forge?

If you don't want to use this MultiMC, or what, like Robin suggested, you can just go to your folders (assuming you are running on Windows), search %AppData% and find .minecraft there, or, go to: C:\Users\YOURUSERNAME*\AppData\Roaming.minecraft , as this is the default location of .minecraft on Windows.

If you are not on Windows (example:Linux), the .minecraft folder is usually located in the "Home" folder as a hidden folder, so you should turn on hidden folder visibility.

So, if you found your .minecraft folder, go to the Versions folder and delete the old Forge version folder.

That's it. I hope it helped :)