SSH without sourcing .bashrc

I was experimenting with my .bashrc file while remotely logged in to a server over SSH. I inadvertently left an exit in there that caused all subsequent logins to immediately disconnect. I was effectively locked out. I was able to recover access with intervention from someone with root privileges, but would it have been possible without help from someone else?

I tried doing stuff like running ssh <server> 'bash --norc --noprofile' and ssh <server> 'mv .bashrc bashrc-backup', and even trying to forcibly overwrite it with scp empty-file <server>:.bashrc. However, all of these options seem to rely on first sourcing the broken .bashrc before running the command, so none of them worked.

It may well be the case that there is no way out of this kind of a situation. But is that by design? Is there a reason for why it's so easy to lock oneself out a system, e.g. just by running ssh <server> 'echo exit > .bashrc'? Are there ways to mitigate this kind of mistake?

As discussed in the other answer, when an SSH client connects to the OpenSSH server, the OpenSSH server will generally start a shell session on the client's behalf, using the user's login shell:

1. If the client requests an interactive session, the server will launch the user's login shell.
2. If the client requests running a command, the server will use the user's login shell to run the command as a shell command.
3. Utilities like scp, rsync, and git which use ssh for transport will request a command to be run on the remote system, so they fall under #2.

If you have something in the remote user's shell startup files which causes the shell to exit, then you will have trouble getting in.

SFTP however is a special case. The OpenSSH server may be configured to support SFTP without launching an external command. If that's the case, then you'd be able to use sftp to connect to the server and delete, rename, or alter the .bashrc file which is causing the problem.

It depends on how the server is configured to support sftp. It can service sftp sessions by launching an external program (named sftp-server). In this case you'd have the same problem getting in that you have with programs like scp. Or, the server can service sftp session by something referred to "internal-sftp", which doesn't require invoking a shell. It just depends on how the particular SSH server is configured.

The reason this happens is because sshd, the server side component, invokes processes using your shell. If it's spawning an interactive shell, it spawns it as a login shell; otherwise, it uses the -c argument to spawn a non-interactive shell to run the command you specify.

All of the operations you specified (specified commands and scp) are non-interactive operations, so normally bash would not load .bashrc, but bash has special-casing for when invoked by sshd so that it invokes it anyway. If you were using zsh, then .zshenv (which is invoked for all shells) would be loaded, but .zshrc (which is only for interactive shells) would not be unless you were specifically loading a shell session.

In this case, if you're using bash, you're out of luck. There isn't any way to invoke an SSH command on the server side without using the shell; this is true even for scp and sftp. Most of the time, you want to use the shell because it sets up things like PATH for various programs, and it allows for a reasonable amount of scripting of complex commands, so OpenSSH always uses it.

This also has some security benefits: if you try to log into a system account that somehow has a valid password but a shell of /usr/sbin/nologin or /bin/false, then you can't do anything, which is probably what the system administrator intended.

There are ways to mitigate this. Many people keep a Git repository of their dotfiles and develop them on one system, then deploy them to others. For example, I always do dotfile development on my laptop. Presumably you'd notice this problem a little sooner if any time you opened a new terminal window it immediately exited, and you might have root access to fix it yourself.

If you need to test a configuration which might lock you out, such as shell configuration or a sudoers change, you can leave one shell (normal or root, respectively) open and then do some testing, so if you break something, you still have a way in to undo it.