High latency, yet ping shows low latency

Solution 1:

What does "latency" mean ?
Are you comparing ICMP echo response times with application processing round-trip times ?

If you are, that would explain it - and also tell you not to compare apples and oranges :)

Solution 2:

can it be you have packet losses [that cause reteransmissions] for instance for larger packets?

maybe try ping -s 1400 address under linux or ping -l 1400 address under windows.

if that does not help try looking at the traffic with wireshark - maybe there are some strange re-transmissions, corrupted packets?

Solution 3:

A discrepancy like that usually shows an issue in the application / server. While it is possible that your ping is different due to QoS / network prioritization, that's not the first place I'd look if you're talking about systems on the same local network.

What's your latency time accessing the service from the same host it is served by? Does that host provide other services faster? Is it under a heavy IO load?