Set which pulseaudio sink is affected by volume control buttons
It sounds like you most likely are experiencing a bug.
If you are not experiencing a bug, notice that switching the default output sink with pavucontrol will only affect programs you start -after- the change has been made. It will not affect programs that are already running.
To affect programs you already have running, each one will need to be switched individually under the pavucontrol "Playback" tab.
Your pulse-audio configuration files are stored under ~/.pulse/
An alternative is to use pacmd list-sinks and pacmd set-default-sink. There's a question related to this