Why is this fish following me?
It means you found the Fish Medallion! Congratulations!
Yeah, the fish that follows you around is completely useless, as far as I can tell. The walkthrough found on the game's FAQ page also says so:
Section 3: The Sea [...]
You will fall down really far into a giant lake of fish. Immediately swim all the way to the right to find a rope. Climb up it to find a treasure chest that holds the Fish Medallion. When you jump back down into the water, a fish will be following you! It doesn’t serve any purpose, but it is cute.