How to ping a server to discover what OS or other software it is running?
NMAP allows you to do OS detection and service identification. I don't know how netcraft does it -- and using nmap could very quickly move out of the 'unobtrusive' category. But you can test it in-house for sure...
I think NetCraft use the Server: header returned by the webserver. No special tools are needed to do this
$ curl -I http://www.microsoft.com | grep "Server:"
Server: Microsoft-IIS/7.0
$ curl -I http://www.apache.org | grep "Server:"
Server: Apache/2.2.9 (Unix)