Does the extra XP gem apply to me or the weapon?

Solution 1:

You don't directly earn XP. Instead, the XP any item you have equipped earn translates directly to your XP. If all of the items equipped are mastered, you gain 0 XP from a battle.

I can't prove this, but my assumption is that you would get a 25% bonus to the final XP tally after a fight.