If I don't like a quest reward can I pick a different one?

Yes in fact! Just quit the game before accepting the reward (use Alt-F4 on PC, quitting to the dashboard on consoles, or otherwise end your game without saving). You'll lose a few minute's progress (until the last autosave) but the mission should still be completed (if you can, make sure you trigger a save before checking the quest reward).

This way you can reroll the item, most commonly to change the element. Before you start pulling out your hair, note that some rewards are always of a certain element. Also unless the weapon is guaranteed to have an element (Maliwan guns or non-Torgue grenades) it may be very hard to find elemental versions of weapons (particularly specific elemental versions) since those parts are just fairly rare, except on grenades and Maliwan guns.

Also note that stat differences will generally be minor. The gun will feature different parts, but certain features like element, bonus crit damage etc make a big difference for some guns; that's when this is handy.

For most quests this should leave you at the point where you can just turn in the quests, but you might lose some progress when the turn-in point isn't a person; I'm not sure about those.