protocol version mismatch -- is your shell clean?

When following the instructions to do rsync backups given here: http://troy.jdmz.net/rsync/index.html

I get the error "protocol version mismatch -- is your shell clean?"

I read somewhere that I needed to silence the prompt (PS1="") and motd (.hushlogin) displays to deal with this. I have done this, the prompt and login banner (MOTD) not longer appear, but the error still appears when I run:

rsync -avvvz -e "ssh -i /home/thisuser/cron/thishost-rsync-key" remoteuser@remotehost:/remote/dir /this/dir/

Both ssh client and sshd server are using version 2 of the protocol.

What could be the problem?

[EDIT] I have found http://www.eng.cam.ac.uk/help/jpmg/ssh/authorized_keys_howto.html which directs that it is sometimes necessary to "Force v2 by using the -2 flag to ssh or slogin

 ssh -2 -i ~/.ssh/my_private_key remotemachine"

It is not clear this solved the problem as I think I put this change in AFTER the error changed but the fact is the error has evolved to something else. I'll update this when I learn more. And I will certainly try the suggestion to run this in an emacs shell -

Solution 1:

One of your login scripts (.bashrc/.cshrc/etc.) is probably outputting data to the terminal (when it shouldn't be). This is causing ssh to error when it is connecting and getting ready to copy as it starts receiving extra data it doesn't expect. Remove output that is generated in the startup scripts.

You can check if your terminal is interactive and only output text by using the following code in a bashrc. Something equivalent exists for other shells as well:

if shopt -q login_shell; then
    [any code that outputs text here]
fi

or alternatively, like this, since the special parameter - contains i when the shell is interactive:

if echo "$-" | grep i > /dev/null; then
    [any code that outputs text here]
fi

For more information see: rsync via ssh from linux to windows sbs 2003 protocol mismatch

To diagnose this, make sure that the following is the output you get when you ssh in to the host:

USER@HOSTNAME's password: 
Last login: Mon Nov  7 22:54:30 2011 from YOURIP
[USER@HOSTNAME ~]$ 

If you get any newlines or other data you know that extra output is being sent. You could rename your .bashrc/.cshrc/.profile/etc. files to something else so that they won't output extra output. Of course there is still system files that could cause this. In that case, check with your sysadmin that the system files don't output data.

Solution 2:

Troubleshooting this problem:

There is a simple way to test if your shell is clean, for an ssh connection: run a command from the ssh connection, rather than starting an interactive shell. The false command will immediately terminate without producing any output, so it is a good test:

bash$ ssh remotehost false
Enter passphrase for key '/home/user/.ssh/my_private_key': 
bash$

If that command line produces any output, one of your startup scripts is to blame:

bash$ ssh remotehost false
Enter passphrase for key '/home/user/.ssh/my_private_key': 
Welcome to RemoteHost!

This system is company property and is provided for authorized use only, 
as set forth in applicable written policies. Unauthorized use is prohibited 
and may be subject to discipline, civil suit and criminal prosecution.
 
Welcome back - You last logged in 16 days ago...
bash$

Another thing to check if you are getting this error is whether rsync is installed and locatable by ssh:

bash$ ssh remotehost "rsync --version"
Enter passphrase for key '/home/user/.ssh/my_private_key': 
rsync  version 3.0.9  protocol version 30
Copyright (C) 1996-2011 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, symtimes

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.
bash$

If rsync is not in the path, you will instead see something like:

bash$ ssh remotehost "rsync --version"
Enter passphrase for key '/home/user/.ssh/my_private_key': 
bash: rsync: command not found
bash$

You can fix this by either installing rsync, or if it is installed but in an unusual location, passing the location to the rsync command line:

rsync -avvvz -e "ssh -i /home/thisuser/cron/thishost-rsync-key" \
    --rsync-path="/usr/local/bin/rsync" \
    remoteuser@remotehost:/remote/dir /this/dir/

Solution 3:

This is commonly caused by your shell's login stuff outputting stuff on a non-interactive shell. You can test if this is the case by doing:

ssh username@host "/bin/true" > testfile
ls -l testfile

If testfile is NOT 0 bytes, then the trouble is that your shell is outputting something. Check /etc/profile, .profile, .bashrc, .cshrc, etc. If it is, you can change it to check if your terminal is interactive and only output text by using the following code in a bashrc. Something equivalent exists for other shells as well:

if shopt -q login_shell; then
    [any code that outputs text here]
fi

or alternatively, like this, since the special parameter - contains i when the shell is interactive:

if echo "$-" | grep i > /dev/null; then
    [any code that outputs text here]
fi

However, if the test file is in fact 0 bytes, then your shell is behaving, but it is possible that you just have a very old version of rsync. You can tell the client end (assuming it is the newer end) to not advertise such a high version that the old rysnc server version doesn't recognize it. You can do this using the --protocol= option. In my case, using --protocol=30 did the trick.

If you are still having trouble, try ssh in as the user rsysnc is connecting with and try running rsync --version to see if the shell can find rsync. If you get something that says command not found, then rsync might not be installed on the machine you are connecting to or it might not be in the path. Rsync does have options for specifying the path of the remote end, read the man page(s).