Can you look at the logs and see why the IP was denied? The likely culprits are too many logins per hour, or too many failed attempts in the period of watching for that.
I finally had the time to look at logs regarding this. I have checked the asterisk/full logs and /tmp/firewall.log. I then greped all the logs I could find for the blocked IP address. I was not able to find any correlation to the block in the logs.
This is still an ongoing issue with our softphone users. It also occurs frequently on the softphone on my mobile phone.
I think there needs to be a way to change the settings as needed to be less sensitive. Also, there needs to be more detailed logging.
Further there is also a need for a better functional description. How does it hook Asterisk? What is it examining to do its business?
This just occurred again today with a Grandstream user. We have this happen frequently with softphone users at their homes.
My softphone on my mobile phone gets locked out all the time as well. I am really frustrated with this. I am ready to just disable it or hack it and fix the settings.
Once marked as an Attacker, a single request in 24 hour period continues you being marked as an Attacker
fpbxattacker all – anywhere anywhere recent: CHECK seconds: 86400 hit_count: 1 name: ATTACKER side: source mask: 255.255.255.255
If you make more than a 100 requests in a 24 hour period you are labeled as a REPEAT
fpbxattacker all – anywhere anywhere recent: CHECK seconds: 86400 hit_count: 100 name: REPEAT side: source mask: 255.255.255.255
50 attempts in an hour also marks you as REPEAT
fpbxattacker all – anywhere anywhere recent: CHECK seconds: 3600 hit_count: 50 name: REPEAT side: source mask: 255.255.255.255
10 requests in 60 seconds, so one every 6 seconds, will get you a shortblock as a REPEAT.
fpbxshortblock all – anywhere anywhere recent: CHECK seconds: 60 hit_count: 10 name: REPEAT side: source mask: 255.255.255.255
Here is how the traffic is filtered:
Chain fpbxfirewall (1 references)
target prot opt source destination
ACCEPT all – anywhere anywhere
ACCEPT tcp – anywhere anywhere state RELATED,ESTABLISHED
ACCEPT icmp – anywhere anywhere
ACCEPT all – anywhere 255.255.255.255
ACCEPT all – anywhere anywhere PKTTYPE = multicast
ACCEPT udp – anywhere anywhere udp spts:bootps:bootpc dpts:bootps:bootpc
fpbx-rtp all – anywhere anywhere
fpbxblacklist all – anywhere anywhere
fpbxsignalling all – anywhere anywhere
fpbxsmarthosts all – anywhere anywhere
fpbxregistrations all – anywhere anywhere
fpbxnets all – anywhere anywhere
fpbxhosts all – anywhere anywhere
fpbxinterfaces all – anywhere anywhere
fpbxreject all – anywhere anywhere
fpbxrfw all – anywhere anywhere mark match 0x2/0x2
ACCEPT udp – anywhere anywhere state RELATED,ESTABLISHED
fpbxlogdrop all – anywhere anywhere
Basically, despite being in the “registrations” or other allowed lists they still have to go through the rate limiting. If user X is compromised and hacker A connects with their creds then hacker A’s IP is now in the “good lists” and they can reach the UCP, etc. If they starting blasting calls at you then skipping those rate limit checks would allow them to just send calls through you at a high rate with nothing to stop them unless you’re monitoring how many calls.
This could have so many implications outside of just fraud charges. They could slow down your PBX. They could put you at capacity with your provider(s) and start having valid calls rejected. They could get you banned by your provider(s).
Just keep that in mind when you’re thinking about ripping this out and doing your own.
Thanks for the info Tom.
There seems to be something inherently wrong with the methodology or algorithms. I am not the only one who sees these blocking problems. I can reproduce this on my mobile phone by going between 4G and my corporate WiFi for instance. Remote Grandstream phones just seem to randomly get locked out without any correlating log entries on the pbx. There seems to be nothing in the Asterisk logs.
Having to log into the pbx and unblock an endpoint is inconvenient at best. I was on a long car trip recently and became locked out. I was not in a position to log into the pbx and fix it. I can tell you first hand, all I wanted to do was rip out the firewall.
The whole point of the firewall is to be able to have remote users safely and conveniently. If I want safe and inconvenient, I can use a vpn or other methods.
I have been working with Asterisk since version 1.2, many years. I know how much probing and hacking goes on. I would like to have better protection. It likely should be built into the Asterisk core. It is just not usable in it’s current state without continuous frustration.
At a minimum, there needs to be logging to ascertain the cause of lockouts and there needs to be settings to control the aggressiveness.
The System Firewall allows you to hook into it with your own rules both v4 and v6. The issue becomes when a single location has more than 6 or so phones at the location and the Internet has issues. If it goes down or lags then numerous requests could end up coming in at the same time, like when the Internet is restored.
You could use a FQDN for those connections and add a regex rule that looks for that domain and you can apply your own rate limit checks for those users that are little more forgiving.
Count me in on this too.
What I can’t figure out is how to whitelist IP addresses in the Responsive Firewall. Even if I add an IP or DDNS address into the “Networks” tab and call it Trusted or Local, Fail2Ban will still block it at random. The correlation between the Networks tab in the Responsive FW, and the Whitelist tab in Intrusion Prevention (System Admin module) needs better documentation. What does each one do? Why do whitelists in the networks tab not seem to work? Etc.
That is because fail2ban sucks. The Firewall, along with the Responsive option, are letting things in and checking them at the interface level, i.e. iptables. Fail2ban is reading logs and looking for X transactions in Y time period.
So while your firewall (iptables) is doing its job right, Fail2ban is going through the logs and saying it sees logged activity that triggers it. So adjust your Fail2ban to not look for either a low count of attempts and/or set a higher threshold.
1 Like
It could me that my reference to Fail2Ban was incorrect. Responsive Firewall is doing the blocking, which I thought runs on Fail2Ban. There is no way to adjust the settings on what logs are searched or what is blocked. I can’t even figure out how to whitelist from the Responsive Firewall, because even endpoints in the trusted and local zones get blocked too when an ATA reboots.
So this is a standard ATA? And every reboot gets it blocked?
Not every reboot. When the NAT router changes the outbound port, it hits the max_contacts, Asterisk rejects the connection, and all of the phones on that external IP get blocked. I click the red X in responsive firewall and the problem is solved until the next time that happens.
Please let’s not conflate any perceived ‘flakey’ fail2ban behavior with any other chains in iptables, the /var/log/fail2ban.log will identify what it is seeing and actioning on. It is an open source python based script that uses regular expression parsing as quickly as your system allows against any targeted log file you point it at. It can be run at any priority in your underlying iptables scheme
If you have badly defined target files or worse, ineffective regexes, it will possibly likely not do what you want, it has effectively protected many systems against many attacks for many years otherwise.
We are not seeing any problems related to fail2ban. These blocks are strictly the Responsive firewall (RFW). The issue is also a problem with softphone on a mobile phones which we use a lot.
I think we could use some settings to adjust the sensitivity of the RFW.
I also found something interesting with a product FusionPBX. It has a feature to block registrations to IP addresses and only allow registrations to the proper domain name of the pbx. This blocks a lot of the riff-raff as they are scanning IP addresses and do not generally know the domain name.
Also, whitelist IP addresses or dns names in RFW by adding them in the network section and setting them in a proper zone. This does work well. It just does not solve the issue for dynamic IP address users who do not have a DDNS name.
1 Like
This is very easy to do on FreePBX (or any Linux-based system). For example, see PBX Security - #7 by Stewart1
I’ve found it to be virtually 100% effective against ‘riff-raff’ (automated tools scanning all IPv4 addresses). Unfortunately, except for the smallest systems (home or micro-business), it’s not an adequate solution. Organizations of significant size have adversaries who will target them explicitly, e.g. a competitor trying to steal a customer list or business plan, or an associate’s wife looking for evidence of his cheating. It’s not hard for an such an enemy to learn the domain name (spearphishing or other social engineering, open Wi-Fi, etc.) Even worse, you may face attackers with specific knowledge of your system (disgruntled employee or former employee).
So, additional restrictions are needed. However, I’m disappointed that the official FreePBX firewall doesn’t also filter by domain name. If you never see attempts from the riff-raff, you can take any attempt that does get logged with the seriousness it deserves. Also, keeping the logs free of scanner probes makes it easier to find and fix problems not related to security.
OK, enough of my rant.
Most NAT routers don’t modify the source port number unless the requested port is already in use. Those that do, e.g. pfsense, have a setting to turn this off. Then, set up all the phones behind a given NAT to use unique local port numbers, e.g. ext. 101 uses 5061, ext 102 uses 5062, etc. The port numbers seen by pjsip should no longer change.
Then, set up all the phones behind a given NAT to use unique local port numbers, e.g. ext. 101 uses 5061
Good idea, but it adds a layer of overhead I don’t want to manage. Anyway, this is besides the point. My point is that the Responsive Firewall doesn’t seem to honor the zones they way I expect them to.
+1 for this… Do you reckon it’s worth submiting a feature request to be able to adjust the aggressiveness of the Reponsive Firewall?
We have the same situation with a remote phone at a home office. Every time their IP address changes, their new address gets blocked. remove the new address from the block list and the phone re-registers fine.
Need to dive into my own logs to have a look but from what @gherbstman is saying, the logs weren’t that helpful. 
We are still seeing this issue. Specifically we have a remote user on DSL. A couple times a month the IP address changes and it gets blocked. It is annoying to have to constantly fix this.
Depending on your ISP and your level of paranoia , you can do a whois lookup of your current IP address as the dslam will always award in IP address within that range. A longer monitoring of the changing address will give you a clue as to how small a network is actually being awarded.
Interesting to know, however I don’t feel over the moon about whitelisting a network of addresses without knowing what’s at the other end of each.
Ho-hum…
Hence the paraonoia level comment. keep a list of the changing ip address dslams usually have only a few thousand customers, and use a much smaller network. Cox and RR have huge networks but in reality use a much smaller DHCP range from any