How to block all traffic to my server that are connecting using my IP instead of my domain name? [duplicate]

If I ask Apache for domain.com that has no Virtual Host setup in Apache configuration, Apache will show the default Virtual Host. How can I prevent this from happening or simply drop the connection?

Use a default virtual host configured to give the response you want in that situation.

Careful what you put there. I've seen clients get upset that we 'stole their domain' when the web host logo came up during some delegation issues =)

This is expected behavior - if you have Apache serving on an IP that a domain resolves to but don't have that domain set up as a virtual host, you get the default virtual host. That's just how it works.