scp \033H message

The fix for this is typically simple. Usually you can inspect .bashrc, find the line or couple of lines near the top that are causing the problem, and move or delete them. The hard part is convincing people that this problem is actually real. Details follow, but if you just want to fix this problem, then you will only need to use this shorter first section.

The Problem, and How To Solve It

This happens when .bashrc in the user's home directory on the remote machine contains commands that produce output and that run even in noninteractive shells. Less commonly, it would also happen if the systemwide /etc/bash.bashrc contains such commands. The specific output varies depending on what is producing it. But the combination of receiving unexpected output and not having any transfers succeed or even start very strongly points toward that cause (especially when the server is a Debian or Ubuntu system). scp uses standard input and output to send and receive data, and if unrelated data are transmitted through them then it cannot transfer files.

If you are thinking, "That's impossible, .bashrc is only for interactive shells!" or otherwise are interested in a detailed explanation of why this occurs, see the second section, below.

This problem does not break normal SSHing. So assuming the system is configured to permit you to ssh in successfully for an interactive login shell, you can do that, open .bashrc in the remote user's home directory, and either remove or comment out (with #) the offending command or commands, if you don't need them. Or if you do need them, then move them below another command that aborts when the shell is noninteractive. Such a command may already be present. In Ubuntu and some other distros, users' .bashrc files and the systemwide /etc/bash.bashrc usually begin with such checks.

The default .bashrc file in Ubuntu, copied from /etc/skel when a user account is created, contains this code to check if the running shell is interactive and to prevent any further commands in the file from running if it is not:

# If not running interactively, don't do anything
case $- in
    *i*) ;;
      *) return;;
esac

Another common method, which some people use in their .bashrc files and which is currently used in the systemwide /etc/bash.bashrc file for all users, is:

# If not running interactively, don't do anything
[ -z "$PS1" ] && return

If the files have been replaced or modified from the default, then you may see either technique use in either file. There are other possible ways to check for interactive operation but they are uncommon. If the user has written their own .bashrc file from scratch or brought it over from another operating system that isn't Debian, Ubuntu, or another Debian derivative, then it is likely that they don't have such code at all. But if you need it, you can still add it.

Any command that produces output and appears in .bashrc or /etc/bash.bashrc, unless it appears after code like that shown above, will cause unexpected data to be sent at the start of an scp session, and will prevent scp from being able to transfer files.

If you recently edited one of these files yourself by adding a command to the top, then it should be easy for you to figure out the specific change that caused the problem. Even if not, the description above may well give you enough information.

However, I recommend that you edit your question with full details, including the contents of those files, whether or not you are able to solve the problem based on the explanation above. Remember that these are the files on the remote server, not the client machine. That should help others who find your question understand the problem, as well as making it possible for more specific advice to be given, if that is needed.

I believe the likelihood of your problem being caused by something entirely different to be very low, but either way, the added information should make it possible to know for sure. Other people than the author of this question who have similar problems and need help to solve them should of course not edit this question, but should post their own question.

Why The Problem Occurs

People often say that .bashrc is only for interactive shells, but that is a misconception, or at best a severe oversimplification. bash runs commands from ~/.bashrc and /etc/bash.basrhc when:

1. the shell is interactive, or
2. another startup script like /etc/profile or ~/.profile sources it, but also
3. when bash is running neither interactively nor as a login shell but determines that it is probably being run as the initial shell in a remote connection.

What bash takes as sufficient evidence that it is a remote shell--and thus whether or not, in practice, this effect even occurs with SSH--depends mainly on how it was compiled, which varies across operating systems, and secondarily on the version of bash and the version of sshd that is being used.

On current Debian and Ubuntu systems, when bash runs as a non-interactive non-login shell, it checks if the SSH_CLIENT environment variable is set and nonempty. (It checks other things too, but for SSH on current Ubuntu systems, they don't reveal anything.) If so, and the SHLVL environment variable is set to a value less than 2--indicating that it is the session's initial shell--bash runs the commands in /etc/bash.bashrc and ~/.bashrc.

To verify this quickly without modifying configuration files, readers may pass those variables manually into bash -c '''s environment and trace what it reads, or examine the run_startup_files function in shell.c (which starts on line 1022 in that version).

A non-interactive non-login bash shell is what you get when you run a script with bash, either by executing it after setting the necessary permissions and giving it an appropriate hashbang line or by running bash your-script explicitly. It is also what you get when you make bash run a one-liner with the -c option, such as:

bash -c 'echo hello world'

As you would expect, the shell you get when you log in via SSH for an interactive session is an interactive login shell. That's what you get when you run a command like this (assuming it succeeds):

ssh [email protected]

But here's where the non-intuitive part comes in: the shell you get when you log in via SSH for a non-interactive session is a non-interactive non-login shell. That's what you get from running a single command via SSH:

ssh [email protected] command args...

That is, running a single command via SSH runs bash as the same kind of shell--a non-interactive, non-login shell--as when you run a single command locally (or within an already-established remote session) using bash -c.

Like ssh in general, scp causes a shell to be run on the remote machine as the remote user. This is still whatever they have configured as their shell, i.e., the shell listed in the user's entry in /etc/passwd or the output of getent passwd (which also gets set as the value of the $SHELL environment variable when they are logged in). Unless they have changed it by running chsh, this is the default user shell, which in Ubuntu is bash.

That's why commands like this run a non-interactive non-login bash shell on the remote server, too:

scp [email protected]:~/test.txt ./

Unless they are guarded by code to stop if the shell is non-interactive, commands in .bashrc or /etc/bash.bashrc are run by such a shell. If they produce output--intentionally or unintentionally--then scp will not be able to copy files.