mod_security with OWASP CRS: Custom rule for whitelisting googlebot

I am about to use OWASP CRS rules with mod_security on my WHM/cPanel enabled CentOS server (with apache). But I fear that accidentally Googlebot may be blocked by one rule or the other. After enabling OWASP CRS, If I add the below custom rule separately, will my rule prevent Googlebot from being blocked by any of the CRS rules. Will the below rule override any CRS rule that may suspect Googlebot.

The proposed custom rule: SecRule REMOTE_HOST googlebot.com$ allow,pass

Update: My website has thousands of URLs and google's crawl rate is between 10K to 50K urls per day. Hence, there is every chance that some brute force rule in OWASP CRS may be triggered for googlebot.

Solution 1:

In the document about veryfying gooblebot, crawlers might come both from googlebot.com or google.com.

Assuming that you don’t check user-agent (easily spoofed), you need to do something like:

SecRule REMOTE_HOST “@rx google(bot|)\.com$” “id:50000,nolog,allow”

Also, this will only work if you enabled the Apache directive HostnameLookups On. Otherwise, you’ll only get IPs.