Yosemite Mail.app is playing sounds for non-inbox mail
My workaround…
Switch off the new message sound in Prefs > General
To the end of every rule, or at least the end of all possible paths, add a new Action - 'Stop Evaluating Rules'.
Add one more Rule at the end, like this…
Anything that gets that far through the Rule list will play the sound, anything diverted on the way will not…
...or you can selectively add that or any chosen sound at the bottom of earlier Rules, for instance if you wanted to differentiate or highlight any particular incoming path.