How to properly configure fail2ban to ban IP if it is accessing some wrong files
Solution 1:
You need to tell fail2ban (via the regex) where in the log entry it will find a <HOST> so that it can then ban that host. For a normal access log that would be at the beginning of the line so
<HOST> -.*(w00tw00t|main.php|setup.php)
would work but may not do exactly what you want as it would match the relevant triggers anywhere in the log entry.
You may want to try something like
<HOST> -.*(/w00tw00t|/main.php |setup.php )
which should tie some of the strings down to more specific locations