Understanding the behavior of C's preprocessor when a macro indirectly expands itself

Solution 1:

Macro expansion is a complex process that is really only understandable by understanding the steps that occur.

1. When a macro with arguments is recognized (macro name token followed by ( token), the following tokens up to the matching ) are scanned and split (on , tokens). No macro expansion happens while this is happening (so the ,s and ) must be present in the input stream directly and cannot be in other macros).

2. Each macro argument whose name appears in the macro body not preceeded by # or ## or followed by ## is "prescanned" for macros to expand -- any macros entirely within the argument will be recursively expanded before substituting into the macro body.

3. The resulting macro argument token streams are substituted into the body of the macro. Arguments involved in # or ## operations are modified (stringized or pasted) and substituted based on the original parser tokens from step 1 (step 2 does not occur for these).

4. The resulting macro body token stream is scanned again for macros to expand, but ignoring the macro currently being expanded. At this point further tokens in the input (after what was scanned and parsed in step 1) may be included as part of any macros recognized.

The important thing is that there are TWO DIFFERENT recursive expansions that occur (step 2 and step 4 above) and ONLY the one in step 4 ignores recursive macro expansions of the same macro. The recursive expansion in step 2 DOES NOT ignore the current macro, so can expand it recursively.

So for your example above, lets see what happens. For the input

EXPAND(TEST PARENTHESIS())

• step 1 sees the macro EXPAND and scans in argument list TEST PARENTHESIS() for X
• step 2 does not recognize TEST as a macro (no following (), but does recognize PARENTHESIS:
  • step 1 (nested) gets an empty token sequence for the argument
  • step 2 scans that empty sequence and does nothing
  • step 3 inserts into the macro body () yielding just that: ()
  • step 4 scans () for macros and doesn't find any
• so the final value for X after step 2 is TEST ()
• step 3 inserts that into the body, giving TEST ()
• step 4 suppresses EXPAND and scans the result of step 3 for more macros, finding TEST
  • step 1 gets an empty sequence for the argument
  • step 2 does nothing
  • step 3 substitutes into the body giving EXPAND(0)
  • step 4 recursive expands that, suppressing TEST. At this point, both EXPAND and TEST are suppressed (due to being in the step 4 expansion), so nothing happens

Your other example EXPAND(TEST()) is different

• step 1 EXPAND is recognized as a macro, and TEST() is parsed as the argument X
• step 2, this stream is recursively parsed. Note that since this is step 2, EXPAND is NOT SUPPRESSED
  • step 1 TEST is recognized as a macro with an empty sequence argument
  • step 2 -- nothing (no macros in an empty token sequence)
  • step 3, substituted into the body giving EXPAND(0)
  • step 4, TEST is suppressed and the result recursively expanded
    • step 1, EXPAND is recognized as a macro (remember, at this point only TEST is suppressed by step 4 recursion -- EXPAND is in the step 2 recursion so is not suppressed) with 0 as its argument
    • step 2, 0 is scanned and nothing happens to it
    • step 3, substitute into the body giving 0
    • step 4, 0 is scanned again for macros (and again nothing happens)
• step 3, the 0 is substituted as the argument X into the body of the first EXPAND
• step 4, 0 is scanned again for macros (and again nothing happens)

so the final result here is 0

Solution 2:

For the purposes of this situation, there are three relevant steps in macro replacement:

1. Perform macro replacement on the arguments.
2. Replace the macro with its definition, with parameters replaced with arguments.
3. Rescan the result for further replacement while suppressing the replaced macro name.

In EXPAND(TEST PARENTHESIS()):

• Step 1, macro replacement is performed on the argument to EXPAND, TEST PARENTHESIS():
  • TEST is not followed by parentheses, so it is not interpreted as a macro invocation.
  • PARENTHESIS() is a macro invocation, so the three steps are performed: The arguments are empty, so there is no processing for them. Then PARENTHESIS() is replaced by (). Then () is rescanned and no macros are found.
  • Step 1 is done, and we have EXPAND(TEST ()). (TEST () is not rescanned because it was not the result of any macro replacement.)
• Step 2, EXPAND(TEST ()) is replaced with TEST ().
• Step 3, TEST () is rescanned while suppressing EXPAND:
  • Step 1, the arguments are empty, so there is no processing for them.
  • Step 2, TEST () is replaced by EXPAND(0).
  • Step 3, EXPAND(0) is rescanned, but EXPAND is suppressed.

In EXPAND(TEST ()):

• Step 1, macro replacement is performed on the argument to EXPAND:
  • Step 1, the arguments to TEST are empty, so there is no processing.
  • Step 2, TEST () is replaced by EXPAND(0).
  • Step 3, this replacement is rescanned, and EXPAND(0) is replaced by 0.
• Step 2, EXPAND(TEST ()) has become EXPAND(0), and EXPAND(0) is replaced by 0.
• Step 3, 0 is rescanned for further macros, but there are none.

The other examples in the question follow similarly. It comes down to:

• In TEST PARENTHESIS(), the lack of parentheses after TEST results in it not being expanded while processing arguments to an enclosing macro invocation.
• Parentheses are put after it when PARENTHESIS is expanded, but this is after TEST was scanned, and it is not rescanned during processing of the argument.
• After the enclosing macro is replaced, TEST is rescanned and is replaced then, but, at this time, the enclosing macro’s name is suppressed.