Using the word "minutes" when saying the time
Solution 1:
As a British academic I use the authority of my life’s experience (perhaps insufficient for this site, but come on guys…) to state categorically but subjectively that the forms are:
Five past X, ten past X, quarter past X, twenty-five past X, half past X, twenty-five to X etc.
Also: X fifteen, X thirty, X forty-five
or
Five minutes past/to, ten minutes past/to etc.
(But not X fifteen minutes etc.)
However for minutes not divisible by five, the word “minutes” is always included e.g.
Six minutes past five, thirteen minutes to three
(but never “six past five” etc.)
One can, however, say:
“X twenty three” etc., although use with two small numbers, especially if consecutive, e.g. “three four” would sound odd. However, as Lawrence commented, you could say “three oh four”. This has a rather military ring — “Synchronize watches gentlemen…”.
Footnote
“Past” rather than “after” and “to” rather than “before” represent the British working class usage that moulded my vocabulary. I was under the impression that “after” and “before” were US usage — but see the comment from @PeterShor.
Solution 2:
The other answers say that in the UK, people use both "ten minutes to six" and "ten to six". Speaking as somebody living in the U.S., the same is true in the U.S. Using "minutes" is slightly more formal, but nobody is going to blink an eye if you say "ten to six" in a formal situation.
Looking at Google Ngrams, there doesn't seem to be a big difference between U.S. and U.K. usage.