Do I need to use a weapon in order to level it, or simply equip it?

I've noticed several times that weapons I have equipped, but not using at that moment, will level up when I get a kill or complete an objective. (My secondary will level up when I get a kill with my primary and vice versa).

So do all of my weapons level up at the same rate once I have them equipped for a mission?

It depends on how you complete the mission and kill enemies.

Affinity (the exp used to level up weapons) is split amongst all your equipped items. When killing an enemy with a weapon 50% of the the affinity goes to the weapon used and 50% to your warframe. You also gain affinity when a nearby ally gains affinity, 25% of the gained affinity goes to your warframe and 75% is split evenly amongst the rest of your equipment. You can read the full rules regarding affinity here.