Can I prevent a driver from being installed with windows 8.1?

You can't disable the automatically installation of drivers for a specific device. But you can choose, that you can choose, what to do, at the device-manager.

Follow this tutorial from wiki-how:

  1. Search for "Change device" in the Windows 8 Start menu. enter image description here
  2. Click on where it says Change device installation settings. enter image description here
  3. Select the No, let me choose what to do option. enter image description here
  4. Check the automatically get the device app option, then click Save changes to save the settings you just chose.

    Important Now check the Never install drivers from windows update option. enter image description here

source

If you can't access the search (like in safe mode), you can open the "Device Installation Settings" Window with the following command:

rundll32.exe newdev.dll,DeviceInternetSettingUi 2