Bug or Feature: Intrusion detection blocks a whitelisted IP

sysadmin
intrusion-detection
firewall
Tags: #<Tag:0x00007f4f421eb4d0> #<Tag:0x00007f4f421eb2c8> #<Tag:0x00007f4f421eb138>

(Jared Busch) #1

FreePBX 15 and Asterisk 18, fully updated system (well last month when I tested) this at least.

Advanced Firewall settings are set to sync to Intrustion Detection and the list shows in Intrusion detection.

Issue: A device that was incorrectly programmed (yes our fault) was failing to register to a valid extension with a password from a whitelisted IP address.

Result: Intrusion detection blocked the IP, taking down the entire site.

I believe there was a thread (or two) in the last few weeks discussing this exact scenario that @jerrm and some others were discussing this, but my search is failing me this morning.


#2

Could it be this thread ?

I also posted for a similar problem in the past:


#3

Yes to: Responsive Firewall always blocks "good" external users

@yois had submitted a PR that was a step in the right direction and did more work after I last looked. Looks like it should be in edge now.


#4

Nice, I will try this.

14.0.3.7: #22196 SIP device with a lot of BLFs can trigger a responsive firewall block


(Jared Busch) #5

Not being blocked by Responsive Firewall, but by Intrusion Detection


(Jared Busch) #6

So I just went to get screenshots and logs to provide clarity and now I am not seeing where iptables is even references the fail2ban info?
If I add something to the custom white list.

I can see fail2ban adding it to its list.
image

But if I then check iptables -L -n -v, i[m not seeing anything referencing fail2ban.
It points the fpbxfirewall chain.

Which only references these chains.

My manually added whitelisted IP never shows up in iptables
image

Edit: For simplicity, I have the register IP, trusted, etc, all unchecked. But if they are selected, I can watch fail2ban add them in its log.


#7

The PR from @yois should go a long way towards fixing if it does what it looks like, but I haven’t tested.

I think I suggested slightly different logic, but that’s the advantage of being the one to submit the PR, @yois can submit it the way he wishes. Not going to nitpick a reasonable fix.


#8

I’ve seen the fail2ban rules “get lost” and not come back before any of the whitelist integration features were implemented.


(Jared Busch) #9

~So where should the fail2ban rules be called? I’m not seeing it at all on multiple updated systems. Do you (or anyone else) have an example of the valid chain?~

Nevermind, I found a system that a client has not updated.

[jbusch@pbx ~]$ sudo iptables -L
[sudo] password for jbusch: 
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
fail2ban-recidive  all  --  anywhere             anywhere            
fail2ban-SIP  all  --  anywhere             anywhere            
fpbxfirewall  all  --  anywhere             anywhere            
[jbusch@pbx ~]$ sudo fwconsole ma listonline | grep "firewall\|sysadmin"
| firewall             | 15.0.8.9   | Online upgrade available (15.0.8.14)         | AGPLv3+     |
| sysadmin             | 15.0.21.28 | Online upgrade available (15.0.21.46)        | Commercial  |

Updated system:

[jbusch@pbx ~]$ sudo iptables -L -n
[sudo] password for jbusch: 
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     udp  --  52.14.37.123         0.0.0.0/0            udp dpt:161 /* Skyetel SNMP */
fpbxfirewall  all  --  0.0.0.0/0            0.0.0.0/0           
[jbusch@pbx ~]$ sudo fwconsole ma listonline | grep "firewall\|sysadmin"
| firewall             | 15.0.8.14  | Enabled and up to date                       | AGPLv3+     |
| sysadmin             | 15.0.21.46 | Enabled and up to date                       | Commercial  |
[jbusch@pbx ~]$ 

(Jared Busch) #10

It is caused by enabling the sync. I disabled it on the updated system and the rules came back.

[jbusch@pbx ~]$ sudo iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
fail2ban-recidive  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-BadBots  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 80,443
fail2ban-FTP  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 21
fail2ban-apache-auth  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SSH  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 22
fail2ban-PBX-GUI  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SIP  all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     udp  --  52.14.37.123         0.0.0.0/0            udp dpt:161 /* Skyetel SNMP */
fpbxfirewall  all  --  0.0.0.0/0            0.0.0.0/0           

But this does not explain how fail2ban would block something then… The client system I found the issue on has not had anything changed. Just has intrusion detection stopped.

I will arrange a maintenance window to turn it back on and verify the issue again, because obviously, with intrusion detection stopped the rules are gone.


(Jared Busch) #11

Created an issue related to the disappearing fail2ban (Intrusion Detection) rules: https://issues.freepbx.org/browse/FREEPBX-22512

Will test more regarding the ID blocking a whitelisted IP.


(Jared Busch) #12

I came back to this today to test the main issue of intrusion detection banning something trusted and the rules were missing, but the system was still set to legacy.

The PBX was rebooted this morning.

I changed it from Legacy to Enabled. Rules still missing.

I changed it back to Legacy. Rules came back.


(Yois) #13

There’s another bug filed for this issue here:
https://issues.freepbx.org/browse/FREEPBX-22543

and I have another PR in the works for that also. The problem is that the original RFW code was written assuming that fail2ban wouldn’t be running alongside it, so it would flush iptables at times, and always INSERT it’s fpbxfirewall rule above all other rules. I’m not quite sure why this only started appearing recently, because all the code I’m looking at appears to have originated from @xrobau in 2016.

As mentioned in another thread, one of the biggest issues plaguing fail2ban in FreePBX is that is VERY OLD. 0.11.1 is out for CentOS, and I have have created an RPM to update the version in the distro here:
Yois / firewall-fix / ae40a6f43e9 - FreePBX GIT

The RPM fixes a bunch of outstanding issues with fail2ban and asterisk fail detection. The main benefit concerning this thread and the bug we’re discussing is that 0.11.1 dynamically creates the INPUT rules too, so it will self heal. There are just a few known issues I’ve discovered:

  • Intrusion Detection will show stopped in the GUI, since the code in the Sysadmin module (which I can’t fix) uses pidof and ps -A to detect if fail2ban-server is running, but 0.11.1 runs asynchronously and will only be detected with pgrep -f, as python is holding the process
  • fail2ban 0.11.1 displays it’s results missing one “TAB” in the list of Jails and IP lists, and the intrusion detection code relies on a “cut -f3” within its hooks in a bunch of places to read that information. In 0.11.1 the information will be in field 2, not field 3.

I’m getting tired and any additional programming help, testing, or buying me a beer will be appreciated.


(system) closed #14

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.