What's the difference between "purge" and "remove --purge"?

Whether they should or shouldn't both exist is perhaps a debating point.

However - they are absolutely identical.

From the man page of apt-get

--purge
           Use purge instead of remove for anything that would be removed. An
           asterisk ("*") will be displayed next to packages which are
           scheduled to be purged.  remove --purge is equivalent to the purge
           command. Configuration Item: APT::Get::Purge.

They key part is --purge is equivalent to the purge command

As to why - I would surmise this is historical -

apt-get --purge remove came before apt-get purge

Looking at the old apt documentation it make reference to the older version of the command. The newer documentation gives the aptitude & apt-get purge example.

For the sake of consistency - its a good idea to not remove old interfaces - if you have an old script - it will still work today because the interface commands still exist.

Mind you that doesnt stop Gnome from deprecating api's - but that's another story...

There isn't one as far as I know, they do the same thing exactly, remove packages and try to remove any configuration files created by those packages.

Basically apt-get purge was implemented after apt-get remove --purge but the old option styled purge still remained.