How to disable the rubber banding/elastic scroll effect in OS X Mavericks?
There is no known solution at the moment – Apple seems to have deprecated the NSScrollViewRubberbanding command in 10.9.
You may find that it helps to do Settings > Accessibility > Mouse & Trackpad > Trackpad Options > Scrolling > Without Inertia. This won't actually disable the rubber band effect, but in my experience it makes it less annoying.
Source: https://discussions.apple.com/message/23778772#23778772
Supposedly Cocktail may eliminate the bouncing, but reports are mixed.