Removing QWERTY Keyboard Layout Permanently
Solution 1:
You should be able to remove the keyboard layout by removing it, but as in your case it can't be done, the second (alternate) option seems more apt at the moment till someone else can figure that out. What I can show you is how to set DVORAK to be your default keyboard layout.
Firstly you need to set DVORAK to be your default input language and apply the changes. This will set it to open all your programs with the DVORAK layout by default.
You can also configure a keyboard shortcut to switch between keyboard layouts or input languages. In your case will be your keyboard layout to left alt + shift on your keyboard. Just in case your layout settings are acting nasty and not setting it to DVORAK by default.