Oh, well, I found the answer myself. It's not a bug, just go into plugin -> alternative toolbar setting, there's un option to turn on volume icon.
No volume icon in Rhythmbox
Oh, well, I found the answer myself. It's not a bug, just go into plugin -> alternative toolbar setting, there's un option to turn on volume icon.