Pick up without swapping weapons

When ever I pick up a new weapon it always swaps me to it, I understand this is probably designed to make me try it out, but is there any way to stop this, or at least make it so that I can choose which weapon it replaces to avoid me having two weapons that do similar jobs, rendering then redundant?

Usually there will be an arsenal very close to the weapon pickup location which will let you change your loadout (just like the one you can build at the Bastion). You are only forced to use the new weapon briefly in most cases, and against enemies that it is particularly suited to.

I don't think that's possible: the tutorial (Menu > How to play) itself explains that when you pickup a weapon, it will automatically replace one of those you are carrying. And since you don't have a "regular" character's inventory, you can't swap easily.

In my opinion, this is a pure developers' choice (and a weakness of the gameplay: you should at least be given the choice of the weapon you're going to replace with the new one).