Assigning programs to specific audio outputs in Windows 7

Solution 1:

Windows 7 automatically transitions open applications to match the default audio device so you don't have to restart your programs to make the change take effect...which is why you're having this problem. but you already figured that out probably.

I'd suggest your headphone's output default so all games use it by default. Then you go into your music player's audio output and change it to be your speaker's output in the program preferences. I've done this with onboard (motherboard) sound and a sound card with VLC media player. Unfortunately I've never done it for any other programs but it's typically an advanced setting/preference, google should help if you query "change audio output for for XYZ".

The reason you're setting the headphones to default is because game's audio options for changing outputs will probably be more limited than music programs. And if you play many games you'll have to change each game versus just one music program.