Disable mouse interference in input source overlay when switching?
Solution 1:
As I've never seen a keyboard with that 'globe' on the Fn key to switch layouts, I'm curious as to what your Control panel for it looks like.
System Prefs > Keyboard > Shortcuts > Input Sources.
If that's anything like mine, just switch it to another key command. I use Ctrl/Spacebar & like you I only have 2 languages, so it's a simple flip between the two.
This produces no screen overlay, but if you have the language menu bar item, you see the flags switch.