Does the order in which you pick up items matter?

No, the order in which you pick up items doesn't matter.

If you don't believe me, let's do some Math!

Now, I can't find the current formula used in Rebirth, but I think that we can still rely on the original Binding of Isaac damage calculation.

In the previous version of the game, the damage formula was something like:

Effective Damage = Damage_Multiplier * (1 + 1.2 * Basic_Damage)^1/2

Each character starts with a base damage of 3.5 multiplied by a character-specific multiplier, for the sake of simplicity we can use 3.5. And we can forget the square root.

Every multiplicative bonus affects Damage_Multiplier, every additive bonus affects Basic_Damage.

For example, Cricket Head gives you:
+0.5 Damage up
x1.5 Damage multiplier

1.5 * (1 + 1.2 * (3.5 + 0.5)) = 8.7

Small Rock instead gives you +1 damage,

1.5 * (1 + 1.2 * (3.5 + 0.5 + 1)) = 10.5

You can see that the result is not affected by the order in which you get an item.

No, the order in which you pick up items does not matter.

According to this Reddit post (/u/Twinge is pretty widely known around the subreddit and is pretty reliable), the damage multiplier is constantly being recalculated as opposed to being calculated once and then forgotten about, so the order you pick up items won't matter.