How to determine if I'm logged in via SSH?

You could use "w" or "who" command output. When you connect over ssh, they'll show your source IP.

Here is a great answer I found on unix.stackexchange:

• If one of the variables SSH_CLIENT or SSH_TTY is defined, it's an ssh session.
• The login shell's parent process can be checked with ps -o comm= -p $PPID. If it is sshd, it's an ssh session.

if [ -n "$SSH_CLIENT" ] || [ -n "$SSH_TTY" ]; then
  SESSION_TYPE=remote/ssh
else
  case $(ps -o comm= -p $PPID) in
    sshd|*/sshd) SESSION_TYPE=remote/ssh;;
  esac
fi

You could add SSH_* to env_keep in sudoers so that this can be detected while switched to the other user.