Change the LodestoneDimension NBT tag from a compass not working

Player NBT can't be modified because the game doesn't allow that. The /replaceitem is an exception to this rule but that won't be of much use to you since you need to get the current Dimension. You could make it so that you have to throw the compass onto a block and then modify the data like this:

/data modify entity @e[type=minecraft:item,limit=1,sort=nearest,nbt={Item:{tag:{waypoint:1}}}] Item.tag.LodestoneDimension set from entity @p Dimension