Firewall version 15.0.13 appears to break custom rules

Hello,

Just updated a couple servers to firewall v 15.0.13 this morning and it seems to object to custom rules being loaded at the top of the INPUT chain:

1626357268: There is an invading rule above us: -s 192.159.66.96/27 -j ACCEPT
1626357268: Resetting iptables

The rules are then reloaded without the custom rules in place.

Rolling back to firewall 15.0.8.14 and restarting the firewall has resolved it for now. I will gather some info and create a Jira ticket shortly.

EDIT: Issue created: [FREEPBX-22653] Custom rules appear broken in Firewall 15.0.13 - Sangoma Issue Tracker

Nate

I have not updated this yet.
Let me see what happens. This is my normal rules flow on firewall 15.0.8.14

[jbusch@pbx ~]$ sudo iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
fail2ban-recidive  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-zulu  tcp  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-BadBots  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 80,443
fail2ban-FTP  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 21
fail2ban-apache-auth  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SSH  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 22
fail2ban-PBX-GUI  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SIP  all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     udp  --  52.14.37.123         0.0.0.0/0            udp dpt:161 /* Skyetel SNMP */
fpbxfirewall  all  --  0.0.0.0/0            0.0.0.0/0           

Upgraded to firewall 15.0.13 and now I see this

[jbusch@pbx ~]$ sudo iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     udp  --  52.14.37.123         0.0.0.0/0            udp dpt:161 /* Skyetel SNMP */
fail2ban-SIP  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-PBX-GUI  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SSH  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 22
fail2ban-apache-auth  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SSH  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 22
fail2ban-PBX-GUI  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-apache-auth  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-FTP  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 21
fail2ban-BadBots  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 80,443
fail2ban-zulu  tcp  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-api  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 80,443
fail2ban-recidive  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SIP  all  --  0.0.0.0/0            0.0.0.0/0           
fpbxfirewall  all  --  0.0.0.0/0            0.0.0.0/0           

The Skyetel rule is a custom rule in the GUI


And can confirm things are broke.

@jerrm I made a change on the networks tab.

Now iptables looks like this.

  1. my custom rule is gone.
  2. hello replicationā€¦
[jbusch@pbx ~]$ sudo iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
fail2ban-recidive  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-api  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 80,443
fail2ban-zulu  tcp  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-BadBots  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 80,443
fail2ban-FTP  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 21
fail2ban-apache-auth  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SSH  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 22
fail2ban-PBX-GUI  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SIP  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-apache-auth  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SSH  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 22
fail2ban-PBX-GUI  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SIP  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SIP  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-recidive  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-api  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 80,443
fail2ban-zulu  tcp  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-BadBots  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 80,443
fail2ban-FTP  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 21
fail2ban-apache-auth  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-PBX-GUI  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SSH  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 22
fail2ban-apache-auth  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SSH  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 22
fail2ban-PBX-GUI  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SIP  all  --  0.0.0.0/0            0.0.0.0/0           
fpbxfirewall  all  --  0.0.0.0/0            0.0.0.0/0           

One sudo fwconsole restart later ā€¦

[jbusch@pbx ~]$ sudo iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     udp  --  52.14.37.123         0.0.0.0/0            udp dpt:161 /* Skyetel SNMP */
fail2ban-SIP  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-PBX-GUI  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SSH  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 22
fail2ban-apache-auth  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SSH  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 22
fail2ban-PBX-GUI  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-apache-auth  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-FTP  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 21
fail2ban-BadBots  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 80,443
fail2ban-zulu  tcp  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-api  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 80,443
fail2ban-recidive  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SIP  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SIP  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-PBX-GUI  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SSH  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 22
fail2ban-apache-auth  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-PBX-GUI  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SSH  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 22
fail2ban-apache-auth  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-FTP  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 21
fail2ban-BadBots  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 80,443
fail2ban-zulu  tcp  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-api  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 80,443
fail2ban-recidive  all  --  0.0.0.0/0            0.0.0.0/0           
fail2ban-SIP  all  --  0.0.0.0/0            0.0.0.0/0           
fpbxfirewall  all  --  0.0.0.0/0            0.0.0.0/0           

It looks like a recent update to drivers/Iptables.class.php in the firewall module added some check to ensure that rules created by the firewall module (containing either ā€˜fpbxfirewallā€™ or ā€˜fail2banā€™) are at the top of the INPUT chain and this seems to be causing the issue.

This where it was added - starts around #1557 but that function has had some updates prior to 15.0.13.

@ngingras - This is my fault, I wrote that change. I updated the JIRA that Iā€™ll try to think of a better way to do this. If you look at older versions of the code, the process of ensuring that weā€™re ā€œon the topā€ was listed as a TODO, and it didnā€™t occur to me that there was a reason this wasnā€™t done.

@sorvani - the duplication issue was known to me but was supposed to be resolved by flushing iptables. I will take another look.

the GUI custom rules also being wiped is an issue since it does get applied with a firewall restart.

I think @yois was the one working on the fail2ban stuff.

Iā€™ve never been a fan of how the module handles fail2ban or restarts in general. I think anything with the current basic structure will be problematic. I donā€™t use the module myself, I donā€™t think Iā€™ve done much with it outside the LE related fixes.

INPUT should be three or so static, NEVER changing rules. F2B should be in itā€™s own chain, custom rules should be in their own chain, the restart unload/reload code has always been bonkers, the call to F2B is likely better placed within the fpbxfirewall chain, we should have a much more current F2B version, I could go onā€¦

100% but changes weā€™re making canā€™t break things either.

1 Like

v16 is arguably the place to make more substantial forward only changes.

Even with the current sng-7 based v16 releases, a decision could be made to go ahead and upgrade custom sangoma rpms like the distroā€™s f2b to the stock RHEL8 version, instead of the ancient version they have, which is older than the stock RHEL7 version. My guess is the current version comes from (or pre-dates) when the distro was centos6 based and they wanted to keep things consistent across FreePBX versions.

Pull Request #126: FREEPBX-22653 Firewall custom rules detection - FreePBX GIT

It turns out that the firewall module takes away custom iptables rules even when it is disabled.

fwconsole firewall disable
fwconsole reload

then,

# iptables -I INPUT -m udp -p udp --dport 69 -j DROP
# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
DROP       udp  --  anywhere             anywhere             udp dpt:tftp
fail2ban-recidive  all  --  anywhere             anywhere            
...

thereā€™s my custom drop rule at the top of INPUT chain.

About 5 minutes pass, then,

# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
fail2ban-recidive  all  --  anywhere             anywhere            
...

Itā€™s gone!

Whatā€™s killing the firewall rule with the firewall disabled and no further intervention from me?

EDIT: I reread your post and this reply isnā€™t relevant, but Iā€™m leaving it anyway for posterity

Read the codeā€¦ I added a check to ensure rogue services wouldnā€™t break the firewallā€¦ I didnā€™t consider custom rules when writing that check. The PR above will allow rules above fpbxfirewall chain that donā€™t contain ā€œfail2banā€ when custom rules are enabled.

I have always preferred the ā€˜restrictiveā€™ approach to fire-walling

Chain INPUT (policy DROP)
.
.
.
(static  rules)
.
.
(dynamic rules starting   fail2ban at the end)
.
.
.

That way you can start or stop or otherwise use the fail2ban-client as needed

Thereā€™s an update to FREEPBX-22653 with a tarball for anyone who wishes to test.

You can install the testing tarball with:

fwconsole ma downloadinstall <url>

You can revert the change with

fwconsole ma downloadinstall firewall --force

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.