How to get Mavericks 10.9.1 to remember external keyboard modifier keys
It sounds like the settings aren't getting saved properly. Try running Disk Utility and Fix Permissions on your startup volume. I usually run this a couple of times and reboot when I'm done when I start having problems like this.
If that fails check the system logs in Console.app while making these changes to see if System Preferences.app is writing any error messages when changing this setting or quitting.
I remap the caps lock key to control and I've never had this problem when switching keyboards, which is something I do every day.
Good Luck.