Why was the space character not chosen for C++14 digit separators?
There is a previous paper, n3499, which tell us that although Bjarne himself suggested spaces as separators:
While this approach is consistent with one common typeographic style, it suffers from some compatibility problems.
- It does not match the syntax for a pp-number, and would minimally require extending that syntax.
- More importantly, there would be some syntactic ambiguity when a hexadecimal digit in the range [a-f] follows a space. The preprocessor would not know whether to perform symbol substitution starting after the space.
- It would likely make editing tools that grab "words" less reliable.
I guess the following example is the main problem noted:
const int x = 0x123 a;
though in my opinion this rationale is fairly weak. I still can't think of a real-world example to break it.
The "editing tools" rationale is even worse, since 1'234 breaks basically every syntax highlighter known to mankind (e.g. that used by Markdown in the above question itself!) and makes updated versions of said highlighters much harder to implement.
Still, for better or worse, this is the rationale that led to the adoption of apostrophes instead.
The obvious reason for not using white space is that a new line is also white space, and that C++ treats all white space identically. And off hand, I don't know of any language which accepts arbitrary white space as a separator.
Presumably, Unicode 0xA0 (non-breaking space) could be used—it is the most widely used solution when typesetting. I see two problems with that, however: first, it's not in the basic character set, and second, it's not visually distinctive; you can't see that it isn't a space by just looking at the text in a normal editor.
Beyond that, there aren't many choices. You can't use the comma, since
that is already a legal token (and something like 1,234 is currently
legal C++, with the meaning 234). And in a context where it could occur
in legal code, e.g. a[1,234]. While I can't quite imagine any real
code actually using this, there is a basic rule that no legal program,
regardless how absurd, should silently change semantics.
Similar considerations mean that _ can't be used either; if there is a
#define _234 * 2, then a[1_234] would silently change the meaning of
the code.
I can't say that I'm particularly pleased with the choice of ', but it
does have the advantage of being used in continental Europe, at least in
some types of texts. (I seem to remember having seen it in German, for
example, although in typical running text, German, like most other
languages, will use a point or a non breaking space. But maybe it was
Swiss German.) The problem with ' is parsing; the sequence '1' is
already legal, as is '123'. So something like 1'234 could be a 1,
followed by the start of a character constant; I'm not sure how far you
have to look-ahead to make the decision. There is no sequence of legal
C++ in which an integral constant can be followed by a character
constant, so there's no problem with breaking legal code, but it means
that lexical scanning suddenly becomes very context dependent.
(With regards to your comment: there is no logic in the choice of a decimal or a thousands separator. A decimal separator, for example, is certainly not a full stop. They are just arbitrary conventions.)
From wiki, we have a nice example:
auto floating_point_literal = 0.000'015'3;
Here, we have the . operator and then if another operator would be to be met, my eyes would wait for something visible, like a comma or something, not a whitespace.
So an apostrophe does much better here than a whitespace would do.
With whitespaces it would be
auto floating_point_literal = 0.000 015 3;
which doesn't feel as right as the case with the apostrophes.
In the same spirit of Albert Renshaw's answer, I think that the apostrophe is more clear than the space the Lightness Races in Orbit proposes.
type a = 1'000'000'000'000'000'544'445'555;
type a = 1 000 000 000 000 000 544 445 555;
Space is used for many things, like the strings concatenation the OP mentions, unlike the apostrophe, which in this case makes it clear for someone that is used separating the digits.
When the lines of code become many, I think that this will improve readability, but I doubt that is the reason they choose it.
About the spaces, it might worth taking a look at this C question, which says:
The language doesn't allow int i = 10 000; (an integer literal is one token, the intervening whitespace splits it into two tokens) but there's typically little to no expense incurred by expressing the initializer as an expression that is a calculation of literals:
int i = 10 * 1000; /* ten thousand */
It is true I see no practical meaning to:
if (a == 1 1 1 1 1) ...
so digits might be merged without real ambiguity but what about an hexadecimal number?
0 x 1 a B 2 3
There is no way to disambiguate from a typo doing so (normally we should see an error)