How can I change audio output device for applications already running?

When I plug in a USB audio device, if no sounds are playing then my output automatically switches to that device. However, if a program is currently playing any audio then it does not switch. If I enter Audio/MIDI Setup and change it manually, [EDIT: or change it in System Preferences -> Sound, or by alt clicking on the volume control in the menu bar] then the running program still does not switch.

[EDIT: I most often encounter this situation with Chrome, VLC, and many video games.]

Is there a way I can switch the output for audio that's already playing, without restarting the program in question?

Solution 1:

If you alt-click on the speaker icon in your toolbar you can select output and input devices.

Solution 2:

It depends on the program that is playing the sound. If I have a youtube video playing in Safari, I can switch between output devices in Audio MIDI Setup, and it works. VLC on the other hand handles the choice of output device for itself, so I have to change it in VLC. The advantage is that I can choose an output device that is not the default one.