Bug or Feature: Intrusion detection blocks a whitelisted IP

[sorvani (Jared Busch)]

FreePBX 15 and Asterisk 18, fully updated system (well last month when I tested) this at least.

Advanced Firewall settings are set to sync to Intrustion Detection and the list shows in Intrusion detection.

Issue: A device that was incorrectly programmed (yes our fault) was failing to register to a valid extension with a password from a whitelisted IP address.

Result: Intrusion detection blocked the IP, taking down the entire site.

I believe there was a thread (or two) in the last few weeks discussing this exact scenario that @jerrm and some others were discussing this, but my search is failing me this morning.

[chrischevy]

Could it be this thread ?

Responsive Firewall always blocks "good" external users

BTW, there is a thread on DSLReports right now about an analog PBX being compromised over a DISA exploit and they got a $33K usage bill. A hard copper line is basically a trusted line. The biggest fault of the carrier? Not reacting to unusual and suspicious call activity.

I also posted for a similar problem in the past:

Trusted network blocked by Firewall

This morning, on one of our system, an IP address that was configurerd as a Trusted Network was added to the Blocked Attackers list. It’s the first time that this happens in like 2 years. Anyone else experimented this issue ?

[jerrm]

Yes to: Responsive Firewall always blocks "good" external users

@yois had submitted a PR that was a step in the right direction and did more work after I last looked. Looks like it should be in edge now.

[chrischevy]

Nice, I will try this.

14.0.3.7: #22196 SIP device with a lot of BLFs can trigger a responsive firewall block

[sorvani (Jared Busch)]

@chrischevy

can trigger a responsive firewall block

Not being blocked by Responsive Firewall, but by Intrusion Detection

[sorvani (Jared Busch)]

So I just went to get screenshots and logs to provide clarity and now I am not seeing where iptables is even references the fail2ban info?
If I add something to the custom white list.

I can see fail2ban adding it to its list.

But if I then check iptables -L -n -v, i[m not seeing anything referencing fail2ban.
It points the fpbxfirewall chain.

Which only references these chains.

My manually added whitelisted IP never shows up in iptables

Edit: For simplicity, I have the register IP, trusted, etc, all unchecked. But if they are selected, I can watch fail2ban add them in its log.

[jerrm]

@sorvani

Not being blocked by Responsive Firewall, but by Intrusion Detection

The PR from @yois should go a long way towards fixing if it does what it looks like, but I haven’t tested.

I think I suggested slightly different logic, but that’s the advantage of being the one to submit the PR, @yois can submit it the way he wishes. Not going to nitpick a reasonable fix.

[jerrm]

@sorvani

But if I then check iptables -L -n -v , i[m not seeing anything referencing fail2ban.

I’ve seen the fail2ban rules “get lost” and not come back before any of the whitelist integration features were implemented.

[sorvani (Jared Busch)]

@jerrm

I’ve seen the fail2ban rules “get lost” and not come back

Nevermind, I found a system that a client has not updated.
```
[jbusch@pbx ~]$ sudo iptables -L
[sudo] password for jbusch: 
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
fail2ban-recidive  all  --  anywhere             anywhere            
fail2ban-SIP  all  --  anywhere             anywhere            
fpbxfirewall  all  --  anywhere             anywhere            
```
```
[jbusch@pbx ~]$ sudo fwconsole ma listonline | grep "firewall\|sysadmin"
| firewall             | 15.0.8.9   | Online upgrade available (15.0.8.14)         | AGPLv3+     |
| sysadmin             | 15.0.21.28 | Online upgrade available (15.0.21.46)        | Commercial  |
```

Updated system:
```
[jbusch@pbx ~]$ sudo iptables -L -n
[sudo] password for jbusch: 
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     udp  --  52.14.37.123         0.0.0.0/0            udp dpt:161 /* Skyetel SNMP */
fpbxfirewall  all  --  0.0.0.0/0            0.0.0.0/0           
```
```
[jbusch@pbx ~]$ sudo fwconsole ma listonline | grep "firewall\|sysadmin"
| firewall             | 15.0.8.14  | Enabled and up to date                       | AGPLv3+     |
| sysadmin             | 15.0.21.46 | Enabled and up to date                       | Commercial  |
[jbusch@pbx ~]$ 
```

[sorvani (Jared Busch)]

It is caused by enabling the sync. I disabled it on the updated system and the rules came back.

[jbusch@pbx ~]$ sudo iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
fail2ban-recidive  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-BadBots  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 80,443
fail2ban-FTP  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 21
fail2ban-apache-auth  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SSH  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 22
fail2ban-PBX-GUI  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SIP  all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     udp  --  52.14.37.123         0.0.0.0/0            udp dpt:161 /* Skyetel SNMP */
fpbxfirewall  all  --  0.0.0.0/0            0.0.0.0/0           

But this does not explain how fail2ban would block something then… The client system I found the issue on has not had anything changed. Just has intrusion detection stopped.

I will arrange a maintenance window to turn it back on and verify the issue again, because obviously, with intrusion detection stopped the rules are gone.

[sorvani (Jared Busch)]

Created an issue related to the disappearing fail2ban (Intrusion Detection) rules: https://issues.freepbx.org/browse/FREEPBX-22512

Will test more regarding the ID blocking a whitelisted IP.

[sorvani (Jared Busch)]

I came back to this today to test the main issue of intrusion detection banning something trusted and the rules were missing, but the system was still set to legacy.

The PBX was rebooted this morning.

I changed it from Legacy to Enabled. Rules still missing.

I changed it back to Legacy. Rules came back.

[yois]

There’s another bug filed for this issue here:
https://issues.freepbx.org/browse/FREEPBX-22543

and I have another PR in the works for that also. The problem is that the original RFW code was written assuming that fail2ban wouldn’t be running alongside it, so it would flush iptables at times, and always INSERT it’s fpbxfirewall rule above all other rules. I’m not quite sure why this only started appearing recently, because all the code I’m looking at appears to have originated from @xrobau in 2016.

As mentioned in another thread, one of the biggest issues plaguing fail2ban in FreePBX is that is VERY OLD. 0.11.1 is out for CentOS, and I have have created an RPM to update the version in the distro here:
Yois / firewall-fix / ae40a6f43e9 - FreePBX GIT

The RPM fixes a bunch of outstanding issues with fail2ban and asterisk fail detection. The main benefit concerning this thread and the bug we’re discussing is that 0.11.1 dynamically creates the INPUT rules too, so it will self heal. There are just a few known issues I’ve discovered:

• Intrusion Detection will show stopped in the GUI, since the code in the Sysadmin module (which I can’t fix) uses pidof and ps -A to detect if fail2ban-server is running, but 0.11.1 runs asynchronously and will only be detected with pgrep -f, as python is holding the process
• fail2ban 0.11.1 displays it’s results missing one “TAB” in the list of Jails and IP lists, and the intrusion detection code relies on a “cut -f3” within its hooks in a bunch of places to read that information. In 0.11.1 the information will be in field 2, not field 3.

I’m getting tired and any additional programming help, testing, or buying me a beer will be appreciated.