When to put "River" before or after its name and why?

Unlike mountain names, where "Mount" always precedes its name, e.g. Mount Everest, I've noticed that some rivers have "River" before its name, e.g. the River Nile but others have it after, e.g. the Colorado River. How does one decide where it should be and how did the difference come about?

There is no such rule, and for that matter, there is no rule about the position of Mount either; there's Rocky Mount in my part of the world, and many Appalachian peaks are known as Nnn Mountain. It is a matter of convention. The same goes for many other geologic or hydrologic features: the Leyte Gulf but the Gulf of Mexico, Loch Lomond but Alemoor Loch, the Isle of Wight but Portsea Island.

That said, Nnn River is the far more prevalent form in the U.S.; it would be quite rare to hear of the River Missouri or the River Columbia in prose. This is also true of other American terms for streams, e.g. branch, brook, run, kills (though not so much for lakes). This format may then carry over to foreign names, unless the entire foreign name is borrowed in whole: thus Americans usually known the Rio Amazonas as the Amazon River, but it is the Arroyo de la Laguna east of San Francisco Bay and not the Laguna Arroyo.

As JeffSahol noted, you can somewhat sidestep the question for rivers by referring to them with the definite article: the Hudson, the Platte, the Potomac.

I suspect British English tends to put river (and the definite article) before the name. It’s always the River Thames, and not Thames River. Mountains are sometimes preceded by Mount, sometimes not. It’s always Mount Snowdon, but it's Ben Nevis (not Mount Ben Nevis) and the Eiger (not Mount Eiger).

Interesting...note that rivers are denoted by the definite article "The", though. The Colorado, as distinguished from Colorado, the state. Mountains' names, without the definite article, tend to need their "honorific" to distinguish them from other proper names or ordinary nouns...with exceptions, as noted below in the comments.

Most rivers in England are "River Foo" but there are exceptions e.g. "Moors River". Streams are always "Foo Stream". Foreign rivers are variously called "Foo River", especially if the author is N American, "River Foo" if the author is e.g. British, but most often just "the Foo" or "the Foo river". This information is gleaned from various book-based searches of river names at Google Books.