Can I get sound out of both iMac and attached Thunderbolt Display?

I have a dual monitor setup (iMac + Thunderbolt Display). The System Preferences > Sound > Output setup only allows me to choose between one or the other monitor. This causes the sound to come only from left or right side.

Any way to get both monitors to play audio simultaneously?

You should be able to use the Audio MIDI Utility (a.k.a. Audio MIDI Setup on Mountain Lion) to create an Aggregate device to mix both output destinations into one item the OS can select.

The Help topics in OSX around this area are...

Play audio through multiple output devices at once

Combine audio devices to use a single device

Yes, it's possible.

By creating an aggregate device, you can combine the audio interfaces of the dual Apple LED Cinema Displays into a single speaker.

Here's an overview of the process:

1. Open /Applications/Utilities/Audio MIDI Setup
2. Click + to add a new Multiple Output Device
3. Select one or more Display Audio devices
4. Adjust the volume slider for each Apple LED Cinema Display
5. Right click on the new "Multi-Output Device" and select "Use this device for sound output" and "Play alerts and sound effects through this device".

There is one caveat to this setup — you can't adjust the volume using either the keyboard shortcut or the speaker icon, as it appears disabled in the OS X Menu Bar^†. you can Option + Click to toggle between audio output devices though.

Sunflower is an open source third-party app that will give you the volume control back, but since it's a kernel extension you should understand the implications before installing it.

Another software to use that isn't a kernel extension and can perfectly overcome the obstacle of not having control over your volume buttons is by using Sound Control. It actually gives you back your volume control buttons on your keyboard. Plus it has a 14 day free trial.