Fail2Ban: Why Fail2Ban is banning Cloudflare ips, but rules seem not match?

fail2ban

This is my /etc/fail2ban/filter.d/nginx-custom.conf

[Definition]

badagents = 360Spider|ZmEu|Auto Spider 1.0|zgrab/[0-9]*\.[0-9a-zA-Z]*|Wget\(.*\)|MauiBot.*|AspiegelBot.*|SemrushBot.*|PHP/.*

failregex = ^<HOST>.*/?XDEBUG_SESSION_START=phpstorm HTTP/1.1".*$
            ^<HOST>.*/vendor/phpunit*.*$
            ^<HOST>.*function=call_user_func_array.*$
            ^<HOST>.*wp-login.php.*$
            ^<HOST>.*wp-content.*$
            ^<HOST>.*/admin//config.php.*$
            ^<HOST>.*github.com/projectdiscovery/nuclei.*$
            ^<HOST>.*owa*.*$
            ^<HOST>.*mifs*.*$

ignoreregex =

This is my /etc/fail2ban/jail.local

[DEFAULT]
ignoreip = 127.0.0.1/8 192.168.1.0/24

[nginx-custom]
enabled  = true
logpath  = /var/log/nginx/access.log
action = iptables-multiport[name=nginx-custom, port="http,https", protocol=tcp, bantime=0]
findtime = 86400
bantime  = -1
maxretry = 1

[sshd]
enabled  = true
action   = iptables-ipset-proto6[name=ssh, port=ssh, protocol=tcp, bantime=0]
findtime = 3600
bantime  = -1
maxretry = 3

This is an example of the log:

188.114.102.58 - - [09/Feb/2021:07:19:16 +0000] "GET /api/annotations?from=1612163956639&to=1612855156639&dashboardId=2 HTTP/1.1" 200 2 "https://www.example.com/d/KUOy7ykRz/domotica-settimanale?orgId=1" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36" "ip.of.my.house"
188.114.102.58 - - [09/Feb/2021:07:19:16 +0000] "GET /api/datasources/proxy/1/query?db=homeassistant&q=SELECT%20mean(%22value%22)%20FROM%20%22one_week%22.%22%C2%B0C%22%20WHERE%20(%22entity_id%22%20%3D%20%27congelatore_6_0%27)%20AND%20time%20%3E%3D%20now()%20-%208d%20GROUP%20BY%20time(5m)%20fill(null)%3BSELECT%20mean(%22value%22)%20FROM%20%22one_week%22.%22%C2%B0C%22%20WHERE%20(%22entity_id%22%20%3D%20%27frigorifero_4_0%27)%20AND%20time%20%3E%3D%20now()%20-%208d%20GROUP%20BY%20time(5m)%20fill(null)&epoch=ms HTTP/1.1" 200 15765 "https://www.example.com/d/KUOy7ykRz/domotica-settimanale?orgId=1" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36" "ip.of.my.house"
188.114.102.58 - - [09/Feb/2021:07:19:17 +0000] "GET /api/datasources/proxy/1/query?db=homeassistant&q=SELECT%20mean(%22value%22)%20FROM%20%22one_week%22.%22%C2%B0C%22%20WHERE%20(%22entity_id%22%20%3D%20%27bagno_5_0%27)%20AND%20time%20%3E%3D%20now()%20-%208d%20GROUP%20BY%20time(5m)%20fill(null)%3BSELECT%20mean(%22value%22)%20FROM%20%22one_week%22.%22%C2%B0C%22%20WHERE%20(%22entity_id%22%20%3D%20%27camera_2_0%27)%20AND%20time%20%3E%3D%20now()%20-%208d%20GROUP%20BY%20time(5m)%20fill(null)%3BSELECT%20mean(%22value%22)%20FROM%20%22one_week%22.%22%C2%B0C%22%20WHERE%20(%22entity_id%22%20%3D%20%270x00158d00044a1f1f_temperature%27)%20AND%20time%20%3E%3D%20now()%20-%208d%20GROUP%20BY%20time(5m)%20fill(null)%3BSELECT%20mean(%22value%22)%20FROM%20%22one_week%22.%22%C2%B0C%22%20WHERE%20(%22entity_id%22%20%3D%20%27salone_1_0%27)%20AND%20time%20%3E%3D%20now()%20-%208d%20GROUP%20BY%20time(5m)%20fill(null)%3BSELECT%20mean(%22value%22)%20FROM%20%22one_week%22.%22%C2%B0C%22%20WHERE%20(%22entity_id%22%20%3D%20%27studio_7_0%27)%20AND%20time%20%3E%3D%20now()%20-%208d%20GROUP%20BY%20time(5m)%20fill(null)%3BSELECT%20mean(%22value%22)%20FROM%20%22one_week%22.%22%C2%B0C%22%20WHERE%20(%22entity_id%22%20%3D%20%27bagno_ospiti_8_0%27)%20AND%20time%20%3E%3D%20now()%20-%208d%20GROUP%20BY%20time(5m)%20fill(null)%3BSELECT%20mean(%22value%22)%20FROM%20%22one_week%22.%22%C2%B0C%22%20WHERE%20(%22entity_id%22%20%3D%20%27cucina_9_0%27)%20AND%20time%20%3E%3D%20now()%20-%208d%20GROUP%20BY%20time(5m)%20fill(null)%3BSELECT%20mean(%22value%22)%20FROM%20%22one_week%22.%22%C2%B0C%22%20WHERE%20(%22entity_id%22%20%3D%20%27balcone_10_0%27)%20AND%20time%20%3E%3D%20now()%20-%208d%20GROUP%20BY%20time(5m)%20fill(null)&epoch=ms HTTP/1.1" 200 57007 "https://www.example.com/d/KUOy7ykRz/domotica-settimanale?orgId=1" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36" "ip.of.my.house"
188.114.102.58 - - [09/Feb/2021:07:19:17 +0000] "GET /api/datasources/proxy/1/query?db=homeassistant&q=SELECT%20mean(%22value%22)%20FROM%20%22one_week%22.%22%25%22%20WHERE%20(%22entity_id%22%20%3D%20%27battery_percentage_myss_camera%27)%20AND%20time%20%3E%3D%20now()%20-%208d%20GROUP%20BY%20time(5m)%20fill(null)%3BSELECT%20mean(%22value%22)%20FROM%20%22one_week%22.%22%25%22%20WHERE%20(%22entity_id%22%20%3D%20%270x00158d00044a1f1f_battery%27)%20AND%20time%20%3E%3D%20now()%20-%208d%20GROUP%20BY%20time(5m)%20fill(null)%3BSELECT%20mean(%22value%22)%20FROM%20%22one_week%22.%22%25%22%20WHERE%20(%22entity_id%22%20%3D%20%27battery_percentage_myss_salone%27)%20AND%20time%20%3E%3D%20now()%20-%208d%20GROUP%20BY%20time(5m)%20fill(null)%3BSELECT%20mean(%22value%22)%20FROM%20%22one_week%22.%22%25%22%20WHERE%20(%22entity_id%22%20%3D%20%27battery_percentage_myss_frigorifero%27)%20AND%20time%20%3E%3D%20now()%20-%208d%20GROUP%20BY%20time(5m)%20fill(null)%3BSELECT%20mean(%22value%22)%20FROM%20%22one_week%22.%22%25%22%20WHERE%20(%22entity_id%22%20%3D%20%27battery_percentage_myss_bagno%27)%20AND%20time%20%3E%3D%20now()%20-%208d%20GROUP%20BY%20time(5m)%20fill(null)%3BSELECT%20mean(%22value%22)%20FROM%20%22one_week%22.%22%25%22%20WHERE%20(%22entity_id%22%20%3D%20%27battery_percentage_myss_congelatore%27)%20AND%20time%20%3E%3D%20now()%20-%208d%20GROUP%20BY%20time(5m)%20fill(null)%3BSELECT%20mean(%22value%22)%20FROM%20%22one_week%22.%22%25%22%20WHERE%20(%22entity_id%22%20%3D%20%27battery_percentage_myss_studio%27)%20AND%20time%20%3E%3D%20now()%20-%208d%20GROUP%20BY%20time(5m)%20fill(null)%3BSELECT%20mean(%22value%22)%20FROM%20%22one_week%22.%22%25%22%20WHERE%20(%22entity_id%22%20%3D%20%27battery_percentage_myss_bagno_ospiti%27)%20AND%20time%20%3E%3D%20now()%20-%208d%20GROUP%20BY%20time(5m)%20fill(null)%3BSELECT%20mean(%22value%22)%20FROM%20%22one_week%22.%22%25%22%20WHERE%20(%22entity_id%22%20%3D%20%27battery_percentage_myss_cucina%27)%20AND%20time%20%3E%3D%20now()%20-%208d%20GROUP%20BY%20time(5m)%20fill(null)%3BSELECT%20mean(%22value%22)%20FROM%20%22one_week%22.%22%25%22%20WHERE%20(%22entity_id%22%20%3D%20%270x00158d00047b5b9c_battery%27)%20AND%20time%20%3E%3D%20now()%20-%208d%20GROUP%20BY%20time(5m)%20fill(null)%3BSELECT%20mean(%22value%22)%20FROM%20%22one_week%22.%22%25%22%20WHERE%20(%22entity_id%22%20%3D%20%270x00158d00032c28fb_battery%27)%20AND%20time%20%3E%3D%20now()%20-%208d%20GROUP%20BY%20time(5m)%20fill(null)%3BSELECT%20mean(%22value%22)%20FROM%20%22one_week%22.%22%25%22%20WHERE%20(%22entity_id%22%20%3D%20%27battery_percentage_myss_balcone%27)%20AND%20time%20%3E%3D%20now()%20-%208d%20GROUP%20BY%20time(5m)%20fill(null)&epoch=ms HTTP/1.1" 200 79394 "https://www.example.com/d/KUOy7ykRz/domotica-settimanale?orgId=1" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36" "ip.of.my.house"

Fail2ban log

2021-02-09 08:19:16,903 fail2ban.filter         [582]: INFO    [nginx-custom] Found 188.114.102.58 - 2021-02-09 08:19:16
2021-02-09 08:19:17,029 fail2ban.actions        [582]: WARNING [nginx-custom] 188.114.102.58 already banned
2021-02-09 08:19:17,060 fail2ban.filter         [582]: INFO    [nginx-custom] Found 188.114.102.58 - 2021-02-09 08:19:17
2021-02-09 08:19:17,078 fail2ban.filter         [582]: INFO    [nginx-custom] Found 188.114.102.58 - 2021-02-09 08:19:17
2021-02-09 08:19:17,631 fail2ban.actions        [582]: WARNING [nginx-custom] 188.114.102.58 already banned
2021-02-09 08:19:17,631 fail2ban.actions        [582]: WARNING [nginx-custom] 188.114.102.58 already banned

Cannot really understand what rule matches... Thank you


Solution 1:

The reason Cloudflare is blocked is because when you use Cloudflare as a proxy, the client requesting the site from your server is the Cloudflare proxys IP address (whois 188.114.102.58 to verify).

You need to restore the clients real IP address by looking at the proxy headers sent from Cloudflare with something like this;

set_real_ip_from 188.114.96.0/20; 
real_ip_header CF-Connecting-IP;

The above is just a snippet, you should read about it on Restoring original visitor IP at Cloudflare. I use a script that pulls the IP's used and places them in /etc/nginx/conf.d/ for automatic inclusion in nginx.

When all of that is done, the nginx access log would instead show the real IP address, not the proxy IP and it wouldn't be blocked by fail2ban.

Related

Why is $upstream_response_time in add_header a hyphen?
Win32_LogicalDisk doesn't retrieve network drives from remote machines
WDS Unattended File (Language and WDS Authentication)
Job for slapd.service failed because the control process exited with error code?
Do IVs affect heroes' maximum stats?
What is/How to use a FO4Edit Merged Patch?
How do I counter a pocketed Bastion?
How do I easily find primitive species planets in my territory?
Can I speed up the AI getting into vehicles
What's the backstory behind those "ghostly" looking animals at Mogg Latan's shrine?
Which multiplayer weapons are available at each rarity level?
What are the quiz questions?