VirtualBox fails to start VM after upgrade from 6.1.26 to 6.1.28 with Result Code: E_FAIL (0x80004005)
Solution 1:
Reading a post reinforced my suspicion that it was the upgrade. Indeed after downgrading back to 6.1.26 (I still had the original download for that version but you can get it from the archive) my virtual machine started just fine.
Not the ideal solution but for now I don't want to waste time with an upgrade that breaks things. If a future version comes up that does work after this error, would be happy to accept that answer.