App Store - Managing Availability of Your Apps' Previous Versions

Edit: this is no longer applicable with the new iTunes Connect UI, check out Sars' answer for an updated version

After looking for this page for a few weeks, I finally figured out how to access it:

1) First, go to the Manage Your Apps section on iTunes Connect.

2) Click the app that you want to edit.

3) On the right-hand side of the resulting page, click "Rights and Pricing":

4) Scroll down and click the terribly-named "Manage iCloud download settings for this app":

5) A list of all versions of your application will appear. You can check a specific version to prevent it from being downloaded due to "a significant usability issue":

Update for new UI of iTunes Connect (2014).

1) Go to "My Apps" section on iTunes Connect.

2) Click the app that you want to edit.

3) On the top menu, click "Pricing"

4) Click "Manage iCloud download settings for this app" at the bottom of window

It looks like the option has moved again.

1. Go to iTunes Connect and select your app under My Apps
2. Select Pricing and Available on the left-hand side.
3. Scroll down to the bottom of the page and expand Last-Compatible Version Settings.
4. Click on the All link to see the available versions.