Why is ping to Google servers and Google DNS so low?

Solution 1:

Google has servers all over the world and uses Anycast so that a server near you answers your request, they use the same method in their public DNS product.

If you have servers on multiple geographic locations, you can set up something similiar with BIND, not using Anycast, but based on geolocation by using views for certain IP addresses.

Solution 2:

ICMP is not the correct tool for measuring network performance. A more realistic test would be using a network packet capture application like Wireshark, or something that that can perform application/protocol-specific tests, such as NPing.

Nping Reference Guide
http://nmap.org/book/nping-man.html

Solution 3:

To explain why it looks like it's a secret, most likely you're not pinging the same server that geoiptool.com sees. You pinged a server in Serbia and geoiptool.com sees another machine in california.

If you want low latency, then place your server closest to an internet exchange in the area where the customers are located. If your customers are spread around the world, then use CDN.