How can I check the verbatim characters of a bash command string?

I had this strange behavior this morning in a bash terminal :

user@home:/home/user$ [ -f /etc/openvpn/client.conf ] && echo true
bash: [: missing «]»
user@home:/home/user$ [ -f /etc/openvpn/client.conf ] && echo true
true

• The first command was pasted from a script edited with gedit.
• The second was typed directly in the terminal.

After some digging, I find out that removing the 30th character (the space between client.conf and "]") and replacing it with a space made the command work again.

My assumption was right : an unknown blank character has slipped into the command, but the question is :

1. How can I reveal those characters in the terminal so that I can debug the command? And more important:
2. How can I prevent this from happening again?

BTW, I running Ubuntu 18.04 / French language, the script that I paste the command from is in a USB drive and may have been edited on Windows too.

Thank you for your very good answers. The bad character is a c2 a0 non-breaking space UTF-8 character. The question How to remove special 'M-BM-' character with sed has interesting fact about that character.

The strange thing is that the script is free of this character. So I don't know where it came from.

Solution 1:

One option is to look at the characters you're trying to use with a hex viewer or editor. hexdump is a good option if you are limited to the terminal.

$ hexdump -Cv <<"EOF"
> [ -f /etc/openvpn/client.conf ] && echo true
> EOF
00000000  5b 20 2d 66 20 2f 65 74  63 2f 6f 70 65 6e 76 70  |[ -f /etc/openvp|
00000010  6e 2f 63 6c 69 65 6e 74  2e 63 6f 6e 66 20 5d 20  |n/client.conf ] |
00000020  26 26 20 65 63 68 6f 20  74 72 75 65 0a           |&& echo true.|
0000002d

You can see here that the space, close-square-brace, space are correct - 0x20, 0x5D, 0x20.

These values are ASCII codes, displayed in hexadecimal. Any value outside the range 0x20 - 0x7E is not a "printable character" as far as ASCII is concerned, and most likely won't play well with command line interfaces.

Note: I copied your first "broken" line for use in the hexdump example above, so something has replaced the not-an-ASCII-space with an ASCII space between your original source and your rendered question.

To repeat this, take the following steps:

1. Type hexdump -Cv <<"EOF" and press Enter
2. Paste the text you would like to use
3. Type EOF on a line of its own, and press Enter

Terminals and Command Line Interfaces don't handle special characters well - as you have discovered. If you aren't very careful with formatting documents, you will also have problems with Microsoft Word (and others) using "smart quotes", em-dashes, the list goes on...

Spot the difference: (the top is "smart quotes", the bottom is "straight quotes")

$ hexdump -Cv <<"EOF"
> “quoted string”
> EOF
00000000  e2 80 9c 71 75 6f 74 65  64 20 73 74 72 69 6e 67  |...quoted string|
00000010  e2 80 9d 0a                                       |....|
00000014

Here, the open quotes are not a simple ASCII quote ("), but are a Unicode / UTF-8 series - 0xE2, 0x80, 0x9C, or U+201C - which the terminal will not handle as you might expect.

Kiwy's suggestion of cat -A also does the job:

$ cat -A <<"EOF"
> “quoted string”
> EOF
M-bM-^@M-^\quoted stringM-bM-^@M-^]$

Note: when using echo "..." | hd, you stand a chance that bash will substitute parts of the string you are trying to inspect. This is particularly of concern when trying to inspect components of a script.

For example try:

$ echo "${USER}"
attie

$ echo "`whoami`"
attie

$ echo "$(whoami)"
attie

$ cat <<EOF
> ${USER}
> EOF
attie

These methods are replacing components with the relevant text. To avoid this, use one of the following approaches. Note the use of single quotes ('), and a "quoted heredoc" ("EOF").

$ echo '${USER}'
${USER}

$ echo '`whoami`'
`whoami`

$ echo '$(whoami)'
$(whoami)

$ cat <<"EOF"
> ${USER}
> EOF
${USER}

Solution 2:

You could use cat with the -A option: from the manual:

   -A, --show-all
          equivalent to -vET
   -E, --show-ends
          display $ at end of each line
   -T, --show-tabs
          display TAB characters as ^I
   -v, --show-nonprinting
          use ^ and M- notation, except for LFD and TAB

So cat -A yourscrip.sh will show you invisible and strange characters.