USB Keyboard won't work after Post but before OS loads
The solution was to enable the "enable legacy USB keyboard support" in the bios.
I think that it actually didn't work the first time like I thought it did. My theory is that what happened then was that there wasn't anything bootable other than the CD so it automatically booted to that.
I'm not sure why I had to do this but it work so I can't complain!