How can I detect if a player reels in a fishing rod?

You can detect the presence of the fishing_bobber entity. For example you could give every player who cast a fishing rod a tag, then do something like this:

/execute at @a[tag=fishing] unless entity @e[type=fishing_bobber,distance=..32] run <command>

Keep in mind that reeling in is not the only way to trigger this, the player can also walk far away (32 blocks seems to be the distance at which the fishing stops), die, etc.