Ubuntu 12.10 xmodmap reset after usb plug/unplug
It took a while to figure out, but I have an answer for this. This actually happens on all Ubuntu versions that I checked and I'm using this fix on 12.04 right now.
I have found that these mappings are not reset if you have all of your xmodmap mappings in a file that is called ~/.Xmodmap.
I had called my file .xmodmaprc and had exactly the behavior that you experience. After changing to the new file name, this annoying behavior went away. My guess is that Ubuntu reads mappings from this file whenever there is a new keyboard added to the system.