Responsive Firewall always blocks "good" external users

firewall
Tags: #<Tag:0x00007fb47d0f6638>

#34

As indicated here, the biggest issue are phones behind dynamic IPs. I can’t recall many issues many behind a static when I used the Firewall module, unless there was a phone with bad credentials.

Most common scenario seemed to be when the IP the phone is behind changes, the phone continues to send packets without realizing it needs to re-register (and therefore clear the IP by the responsive firewall). By the time is re-registers, the IP has been blocked. Particularly a problem if there are multiple BLFs generating a lot of packets, or if there are multiple phones behind the IP, or both.

Increasing the fail2ban retry limit may help, but the freepbx firewall rate-limit/attacker checks may also block the traffic. Unfortunately those limits aren’t tunable.

Mobile softphones on mobile data are generally aware of when their IP changes and register pretty quickly. A crude workaround can be limiting the softphone to mobile data so it is always aware of IP changes, and (mostly) eliminate the multiple phones behind one wifi IP problem.


(Tom Ray) #35

Notice? It is a planned company trip. This would be part of the planning.


(Yois) #36

We are covering all possibilities… Let’s think unplanned trip.


(Tom Ray) #37

Well for a single employee, this is what VPNs are for. More secure too.


(Yois) #38

Most common scenario seemed to be when the IP the phone is behind changes, the phone continues to send packets without realizing it needs to re-register (and therefore clear the IP by the responsive firewall). By the time is re-registers, the IP has been blocked. Particularly a problem if there are multiple BLFs generating a lot of packets, or if there are multiple phones behind the IP, or both Increasing the fail2ban retry limit may help

This. Just… It’s not fail2ban. It’s PHP code listed above. I’m seeing that even without IP changes I have problems every so often, but you’re correct that I’m approaching the code from the wrong vantage point. Having a known IP bypass the RFW is important but won’t solve the issue we’re all having.

I think that the first step to any of this is to enable better logging of what packets are tripping the RFW.


(Yois) #39

OK, I had an idea and made changes to the firewall code, but cannot test it because the file is not signed and the firewall won’t start.

Dev team - how are community members supposed to test firewall code?


Firewall update
#40

I don’t know the proper answer, I just hacked around it when I was doing the certman firewall fixes.

I pointed incron at a bash wrapper script to handle the firewall hooks and passed anything else through to sysadmin_manager.

IIRC, I also had to self sign the module, but standard self-signing didn’t work. I had to copy the signatures from /etc/freepbx.security over the module folder’s sig file or some such nonsense. It’s hazy now and I didn’t make notes.

It was an ugly hack, but was functional enough I was comfortable submitting the PR without waiting for a real answer (or even asking - I guess I should have).

I hope there’s an easier, proper way.


(Yois) #41

Right, self signing the module doesn’t work.

So I just hacked validator.php and commented out the sig checking rules, but I can still only load the PHAR directly with PHP, I can’t get fwconsole or the GUI to load the firewall.

What’s so strange is that despite all of the changes I made nothing is effective. I’m quite certain that this is all due to signature issues. The files are definitely changed and I rebuild the PHAR file, and all of the rules are still unchanged.

I see your le cert rules, I’m just doing the same thing but making the rules more sane.


#42

Keep hacking. Did you try copying over the module sig?


(Tom Ray) #43

There is a whole development outline and how to properly write, update and do bug fixes of code (and even submitting new modules) right here:

https://wiki.freepbx.org/display/FOP/Developer+Corner+Home


(Yois) #44

Ty, this doesn’t apply to firewall as it has an internal validation mechanism to verify signatures.

As @jerrm suggested, I kept hacking and bypassed the validation by commenting out line 55 in validator.php before compiling the phar.

I’m testing my changes but I must have a syntax problem… Will keep everyone posted.


(Yois) #45

OK, so I have this tested and working on my system. The pull request is here:
Pull Request #92: FREEPBX-22170 - Prevent false positives in RFW - FreePBX GIT

Essentially, what I did was add a 90 second bypass to the responsive firewall for the first time we see a packet from an IP address. While this functionality was supposed to be there previously, the existing code would not remove successful registrations from the whitelist, and therefore the second time the whitelist would need to be used it wasn’t available and would cause a false block.

I also fixed the issue that registrations hitting the RFW were not always appearing in the UI, by marking the packets earlier in the chain so that they show up as soon as they hit the RFW, not after they fail it first.

What is still unsolved is that fail2ban is working alongside the RFW, and the rules in fail2ban precede RFW. Because of this, a misconfigured devices will cause FAIL2BAN, not RFW, to DOS an entire site. While I can understand the security involved, the likelihood of a sleep deprived sysadmin mistyping credentials is so much more likely than an inside job attack, that I would like to see fail2ban either disabled once RFW is enabled, or that the ruleset should be further down in the IPTABLES chains so that packets allowed through RFW should not be blocked by fail2ban.

Awaiting community feedback if these changes are smart or not.


#46

I’ll try to fire up a distro image. Need to see the generated rules.


(Alex Brainin) #47

Why not use an encrypted VoIP tunnel for that? For example, with Ringotel, you’ll need to white-list only one IP address in your Firewall and no problem connecting any number of mobile/remote users from any public network. You’ll need less than 10 minutes to set it up.


(Simon Telephonics) #48

Unless I’m mistaken about the purpose, the whole point of the responsive firewall is that you don’t need to use VPNs and whitelists and you can have your system open to remote and mobile users without getting clobbered by every scanner that roams the Internet.


(Alex Brainin) #49

Sure, you can leave that working, just never had an issue with blocking “good” mobile and remote users.


(Tom Ray) #50

Well part of this issue here was that a location with multiple users and a dynamic IP was hitting rate limits because of too many requests from the same IP in X time period.

The logic, as I’m seeing it here, is that as long as the IP has a good SIP authentication then it shouldn’t be rate limited at all. Meaning if account 1 on the random IP auths fine, then all other requests from that IP aren’t checked for Y requests in X time. Per the example of 20 employees all ending up at the same hotel on a spur of the moment trip no one could plan for. Otherwise the hotel IP would be block and they all couldn’t connect.

I feel that is flawed logic.


(Yois) #51

Can you explain why?


(Tom Ray) #52

I have but I will do a quick refresher.

SIP credentials are compromised in some fashion. Bad actor starts sending calls and successfully auths. Now bad actor can go crazy because nothing outside of “concurrent calls” will stop them. So sure they get 3 calls going and then all the rest are rejected until those 3 calls complete. So during those 3 calls the system is just being hit over and over and over again. Rinse and repeat. Basically, if the extension has a high concurrent call setting or is unlimited then nothing is stopping those calls from just going through all the time.

Oh and the added bonus, the IP is also allowed to now just hammer other accounts in a brute attack to try to bust those creds and then start using other accounts to make calls.

I mean, it could end up being really bad.


#53

Or one phone with more than a couple of programmed BLFs behind a dynamic IP.

Fail2ban should still trigger in that scenario.

As @yois suggests, there could be better coordination between the firewall daemon and fail2ban. Even with @yois fix, f2b could still kick in and block good traffic if there is too much bad stuff.

Arguably, the firewall daemon could subsume all of the fail2ban functionality for SIP to possibly allow for more awareness.