In your last example,
echo "$(echo '!b')"
the exclamation point is not single-quoted. Because history expansion occurs so early in the parsing process, the single quotes are just part of the double-quoted string; the parser hasn’t recognized the command substitution yet to establish a new context where the single quotes would be quoting operators.
To fix, you’ll have to temporarily turn off history expansion:
set +H
echo "$(echo '!b')"
set -H