Parameter expansion (variable expansion) and quotes within quotes

While working in a (Unix) shell, sometimes I need to double-quote single-quotes or vice versa, i.e. to write something like "foo 'bar baz' moo" or 'foo "bar baz" moo'. This may be because

  • the resulting string will be parsed again (like with eval, with a shell run by watch, with a shell run by sshd on some remote server etc.) and these inner quotes are crucial to distinguish actual arguments during this second parsing;
  • or I just need the original argument to include quotes for whatever other reason.

Some ambiguity appears when instead of bar baz I have a variable, say $variable. This is what POSIX says:

2.2.2 Single-Quotes

Enclosing characters in single-quotes ('') shall preserve the literal value of each character within the single-quotes. A single-quote cannot occur within single-quotes.

2.2.3 Double-Quotes

Enclosing characters in double-quotes ("") shall preserve the literal value of all characters within the double-quotes, with the exception of the characters backquote, <dollar-sign>, and <backslash>, as follows:

$
The shall retain its special meaning introducing parameter expansion […], a form of command substitution […], and arithmetic expansion […].

[…]

It's not exactly obvious which rule applies for $variable inside single-quotes inside double-quotes (or inside double-quotes inside single-quotes), therefore I'm asking:

  1. Which rule applies to:

    • variable=123
      echo "'$variable'"
      
    • variable=123
      echo '"$variable"'
      

    (I'm aware this can be easily answered just by trying, it would be a reasonable research effort. I decided to put this effort into the community wiki answer below).

  2. Are there any exceptions, quirks, surprises?

    (Please note there is a community wiki answer below. Instead of adding a separate answer with another exception or quirk, consider contributing to the community wiki one).


Solution 1:

Which rule applies?

Just by trying:

  • $ variable=123
    $ echo "'$variable'"
    '123'
    $
    
  • $ variable=123
    $ echo '"$variable"'
    "$variable"
    $
    

So the outer quotes matter. Common shells behave in the same way: sh, bash, dash, zsh, … It's because of this general rule:

If the current character is <backslash>, single-quote, or double-quote and it is not quoted, it shall affect quoting for subsequent characters up to the end of the quoted text.

(source)

Only the outer quotes are not quoted, the rule applies to them. The inner quotes are quoted, they do not affect subsequent characters.


Are there any exceptions, quirks, surprises?

(Well, echo may introduce few surprises, see Why is printf better than echo? Still, this is a separate issue. For readability the answer uses echo with a safe value of $variable).

  1. The above code gives us either '123' or "$variable". What if we need "123" or '$variable'?

    • To get "123":

      variable=123
      echo "\"$variable\""
      #     ^^         ^^   escaped inner quotes
      

      or

      variable=123
      echo '"'"$variable"'"'
      #    1 12         23 3   outer quotes (numbered pairs)
      #     ^  ^^^^^^^^^  ^    quoted fragments
      
    • To get '$variable':

      variable=123
      echo "'"'$variable'"'"
      #    1 12         23 3   outer quotes (numbered pairs)
      #     ^  ^^^^^^^^^  ^    quoted fragments
      
  2. The question notices that $ retains its special meaning in double-quotes. This is important:

    The input characters within the quoted string that are also enclosed between $( and the matching ) shall not be affected by the double-quotes, but rather shall define that command whose output replaces the $(…) when the word is expanded.

    (source)

    It means quoting within $(…) should be considered independently, despite the outer double-quotes. Indeed:

    $ variable=123
    $ echo "$(echo 'foo $variable')"
    $ #            ^             ^   single-quotes matter!
    foo $variable
    $
    

    (Note echo $(stuff) is bad code in general; it's here to illustrate the current issue).

    In these two cases:

    • echo "'$variable'" (tried way above)
    • echo "$(echo 'foo $variable')" (discussed here)

    there is $variable inside single-quotes inside double-quotes, but $(…) changes a lot. Similar situation with echo "`echo 'foo $variable'`" (although $(…) and backticks are not exactly equivalent).