Is it possible to limit SSH client users to connect ONLY if I manually type the key into the SSH Server?

So basically I want to get a more secure way to ssh into my server, because it is located in a very big LAN. I tried using firewall to only allow certain IP's into it, it works, but someone could manually change the IP to the one that is allowed un the UFW.

What I want is that if someone wants to log in the SSH I, as a server, will have to manually input the client's key into the server.

What I want is that if someone wants to log in the SSH I, as a server, will have to manually input the client's key into the server.

That's basically what key based authentication does.

You add a key (or multiple keys) per user that's allowed to connect. The keys are large cryptographic secrets, that's totally unfeasible to brute force. Someone without the key is unable to authenticate.

This is perfectly acceptable to servers exposed to the internet.