Can't remote login after startup

I have an iMac running Catalina (10.15.2) that I don't have a keyboard or mouse plugged into. I just want to use it via screen sharing and SSH.

The problem is that often when I want to use it I can't. SSH hangs, screen sharing won't connect. I've tried restarting it via the power button, but even when it gets to the login screen I can't use SSH or screen sharing.

The only thing that seems to work is plugging in a keyboard and mouse, and logging in the normal way. Then SSH and screen sharing start working. They continue to work for a while. Eventually I come back and it's back in a bad state.

In Energy Saver settings I checked "Prevent computer from sleeping" and "Wake for network access" as well as "Start up automatically after power failure", power nap and hard disk sleep are unchecked.

Solution 1:

This is caused by having FileVault enabled in the security settings. The disk won't decrypt until someone logs in, so services like remote desktop and sshd won't start until then.