NVMe M.2 not showing up - unless insert drive while running
It turns out the solution was to upgrade the power supply. Perahps my 250 Watt power supply was enough power to start the motherboard, but not enough to power the M.2 drive from a cold boot.
ASRock tech support was very helpful. They suggested this as a solution right away, and it fixed my problem.
Edit: Commenter offers this analysis:
... the anemic power supply couldn't supply enough power during start up, with or without the drive, and the drive was just the most sensitive to the issue.