How to escape history expansion exclamation mark ! inside a double quoted string?
Solution 1:
In your last example,
echo "$(echo '!b')"
the exclamation point is not single-quoted. Because history expansion occurs so early in the parsing process, the single quotes are just part of the double-quoted string; the parser hasn't recognized the command substitution yet to establish a new context where the single quotes would be quoting operators.
To fix, you'll have to temporarily turn off history expansion:
set +H
echo "$(echo '!b')"
set -H
Solution 2:
This was repeatedly reported as a bug, most recently against bash 4.3 in 2014, for behavior going back to bash 3.
There was some discussion whether this constituted a bug or expected but perhaps undesirable behavior; it seems the consensus has been that, however you want to characterize the behavior, it shouldn't be allowed to continue.
It's fixed in bash 4.4, echo "$(echo '!b')"
doesn't expand, echo "'!b'"
does, which I regard as proper behavior because the single quotes are shell syntax markers in the first example and not in the second.