I freaquently have a lots of access from google proxy. It says it is Google Favicon bot and I've checked it by host command. User-agent is like following.

"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.75 Safari/537.36 Google Favicon"

I've tried to block it by few lines in robot.txt but no luck.

User-agent: Google Favicon
Disallow: /

I have many web sites and each have sub domains under one domain and the bot accesses all sub domains at once.

It's weird that the log has my company's ip as x-forwarded-for (and google proxy's ip). Yeah, we use Google Analytics monitoring those web sites and every day login to GA console by hand but no one request Google to get all favicons.

Does anyone know how to stop it?


It is one of Google's official crawlers:

https://developers.google.com/search/docs/advanced/crawling/overview-google-crawlers

However, as mentioned in Google's documentation, it comes in response to user-initiated requests (eg. bookmarking a page in your browser is user-initiated), thus it ignores robots.txt directives.