How can I use Dirvish without enabling root logins in sshd?
For anyone looking here's the solution I went with.
Key points:
Dirvish doesn't support running rsync via sudo (Why, I do not know)
In AUTHORIZED_KEYS you can restrict the conditions under which a key is authorized, including specifying a single permitted command, and a single source IP address.
In sshd_config you can specify that ssh root login is only permitted when the -command flag is specified and the command matches the one in AUTHORIZED_KEYS
PermitRootLogin forced-commands-only
A full procedure is here: http://apt-get.dk/howto/backup/
You are on the right track with not allowing root logins via SSH. However your prerequisite about sudo is unreasonable.
However, sudo allows very fine grained control over exactly what can and cannot be executed by a given user. You can allow the dirvish user to ONLY execute exactly one binary without a password.
Combine this with key-based authentication for that user, and you have yourself a backup solution.