How to fix blinking underscore on boot from USB flash drive?

Plop Linux to the rescue!

That'll add a lot of life to legacy machines whose BIOS can't handle booting from a USB. It's easiest if you can burn it to a CD or DVD and boot Plop from that. That will allow you to redirect to your USB device. If you don't have a CD, there are other ways, including using a floppy or, if you have an OS installed on a bootable drive, adding an entry for Plop to your boot manager. You can also just install another hard drive (or use a new partition on an existing drive) and install Plop to there. Any of those will allow you to boot Plop and then redirect to the USB device.

Check out youtube for videos like this one.

Turns out certain file-system setups make this desktop detect the flash drive as a hard drive.
So pretty much it wasn't detected as a removable device, and instead a hard drive by the BIOS. Perfectly able to boot into installation media now by taking the different boot device path no problem.