Solution 1:

I think it's crossfading.

See if it's turned on in your preferences, I was having the exact same problem.

If it is turned on, turn it off and then restart Rhythmbox.

Solution 2:

I've logged a bug, it's seems the right thing to do, I've seen this issue around, if this bug affects anyone else, subscribe here: https://bugs.launchpad.net/ubuntu/+source/rhythmbox/+bug/666199

Solution 3:

It's distortion that's coming from the volume conrol in Rhythmbox. Turn down the volume in the application and you're good to go!