Can the term "G-Man" be used to describe a Government official who is not an FBI agent?

You note a reasonable argument for why "G-Man" might apply to any government agent.

However the term is dated and anyone who does understand it will expect "G-Man" to refer to an FBI agent.

I do have some experience with abuse of the term since my mother thinks it's funny to apply it to any government employee. In my experience people conclude that if she's not referring explicitly to FBI agents she must be referring to federal government agents engaged in a similar role: I.e., plain-clothes field law enforcement. So this does not cause too much confusion if it includes, for example, Secret Service or postal inspectors. But spies or SEC investigators are not typically contemplated.