Is there any reason not to use silenced firearms whenever the option is available?

In Spec Ops: The Line, "secondary fire" for the M9, M4, and Micro-Uzi (and probably others) allow you to fit a silencer to the barrel of your weapon.

Is there any reason not to have these equipped all the time? For example, does the silencer reduce damage, range, movement speed, or anything?

There are parts in the game where you can take a brief Stealth Approach (Walker always says to be quiet in these parts, either way it's your choice and these parts are really really short).

You can order Lugo or Riggs to shoot them stealthily or you can do it yourself with your silencer.

You'll only get detected if some other guy sees the dead body or sees him fall down.

As far as I noticed and read on forums the Silencer reduces Damage a little bit and also reduces the Recoil of the weapon.