Why do we use a French term for a currency-exchange office?

In British English and across Europe, the term Bureau(x) de change is used to describe what US English speakers would call a Currency Exchange or Foreign Exchange (office).

Why do we use a French term - or 'loan word' - for something which doesn't appear to be intrinsically French in nature or origin and is a relatively new concept, so has no historical significance or tradition?

I find it hard to believe that English speakers in Europe struggled to succinctly describe the concept of an office where you can exchange currencies and had to adopt the French term instead?

Perhaps the French term was picked up by English travellers visiting French speaking places (France and Switzerland), where the local language to describe their exchange offices. My guess is most of the tourists would change their money whilst abroad. The term then stuck and was by businesses used back at home, as the association of foreign-ness is appropriate.

Here's an Ngram of the terms in the English corpus. The capitalised and lowercase terms are roughly on par, and you can see the term becoming more popular.

And from the French corpus unsurprisingly has a larger scale, and starts earlier. Interesting to note that the capitalised term doesn't show up.

Checking some old books, bureau de change can be found in English 19th century travel guides for France and Switzerland, and also for money changers in London.

1858's The ABC, or, Alphabetical railway guide has adverts for S. J. Spry And Co. of 110 Leadenhill Street, London who are:

Money Changes, Foreign Coin and Bullion Merchants
Geld wechsel comptoir. Bureau de change.

The Welcome guest: Issues 37-75 of 1859 describes Switzerland:

So many large white buildings, too, with jalousied windows, on whose entablatured friezes you might read " Banque de Commerce," " Banque du Landgrafschaft," " Banque d'Escompte," " Banque et Bureau de Change."

A practical Swiss guide, by an Englishman in Switzerland of 1860 contains several Bureau de Change amongst its listings, such as:

Nouveautes, Silks, Shawls, &c. — Cluzeau Aine, opposite the Post.
Bureau de Change — Uytborck, 74 Montagne de la Cour : of the highest character. Capital of Belgium — a smaller Paris. Manufact. lace. English Embassy : on the Royal Park.

The Bankers' magazine, and statistical register, Volume 19, Part 1 of 1864 translates some financial terms between English, French and Italian:

Exhange office - bureau de change, m. - casa de cambio, f..

Charles Dickens' 1885 journal Household Worlds appears to mention it several times, including these descriptions:

... a "Bureau de Change" stood temptingly before him ; English notes and coins displayed in profusion in its wire-protected window, affording ample evidence that a large business was done in the current coin of that realm.

Both letter and "At Home" were dated from the same house in Regent Street, but the forer bore the business heading, "Bureau de Change."

I bet the real reason is because French used to be considered an international diplomatic language, the way English is now. Indeed American passports are still in both English and French.

Plus, France is the country closest to England, so changing pounds to francs was historically common. This may help explain why the practice is more common in the UK than Stateside.