Solution to help you be protected from distributed sip brute force attacks
Fail2ban is a great tool helping you be protected from normal brute force attacks, when an attacker just brute forces your sip accounts from one ‘evil’ ip address, which, depending on your fail2ban settings, gets blocked really quickly.
I’m looking after an environment of more than hundred FPBXes, lately I noticed that with password brute force attacks the IP of attacker NEVER repeats in ‘wrong password’ logs, meaning every new attempt to crack your password comes from new IP address. With additional analysis I figured out that botnet may have around 1500 different IP addresses at this point of time and basically this beast attacks your SIP server every time from new IP. Fail2ban just can’t block it.
I managed to find a logical solution to block this from happening. Apart from locally blocking attackers on individual FPBXes, I’ve added central automated solution, which, if it sees more than 10 repeats of same blocked IP on all the FPBXes - it blockes it on the firewall on the datacentre edge.
Feedback, thoughts, improvements - welcome. Contact me for help.
Thanks for sharing. Code that’s shared here requires that licensing details be explicitly stated. If you don’t know or care about such things, you can just use the GNU GPL license that FreePBX uses. Please update your repo with a license file.
Thank you for the reminder of getting proper licence, now it’s there.
I saw previously that solution, there is always prons and cons involved with any solution. Basically this one allows to build your own black list for whole your environment. Furthermore attacker will be banned next minute, where as with that service it may take days/weeks, worse situation if only specific environment is targeted.
Have a play, As the script is only few weeks old definitely it may need some tweaks and logic extensions. Don’t mind of working together and making a proper integration to a FPBX module.
Also can my first post be moderated and a proper link added?
I’m seeing about 80-130K attempts a week on 20 boxes or so, and they are the same IP across completely different networks, locations etc. I was debating writing something to receive the email notifications to build a list but unless I’m opening up a division of the Internet Police to go bang some heads, not sure what I would do with the list.
Will look at what you did…
One other note: many of the attackers came off of VPN service providers which I discovered by doing reverse dns lookups…
Just to share, your script found a good amount of IP’s so I went through them and many were already detected by fail2ban.
Part of that may be that I tweak my settings to increase the amount of find time to search for, the ban time is increased and lastly I reduce the amount of max retries.
I also do see the sample of IP’s across different servers as you can see the notifications
Thanks for the feedback, my script uses uniquely developed logic to detect “badboys” - it queries internal FreePBX database for configured SIP/PJSIP extensions. Next, if it finds an attempt to register non-existing extension - the script blocks that fellow. Nowadays Fail2ban can detect majority of dumb brute force attacks, although what I find with my research - some botnets do a fancy logic - they rotate IP they attack from, so it’s not repeated for a day or two in your logs. F2b cannot detect those at all. This script closes that gap.
By the way in my environment script runs every minute on each box, and the day worth of ‘badboys’ offloads to central firewall in the evening. This approach in 2 days reduced volume of attacks by 95%
Later I’m planning to update script if I get more feedback, so for anyone interested - you may click ‘watch’ in GitHub to get updates.
First, thanks for sharing the app!
I agree with your ideas, it works, it would also detect these bad actors more quickly than f2b with my extreme settings.
I have a couple of questions / ideas.
What happens when I delete an extension on the PBX, and the phone is still plugged in trying to login, that would get flagged, but are you looking at the whitelist from within FreePBX to ignore it?
If it’s on the internal network, I know you’re blocking 1IP, so nothing dramatic, but
if your PBX is on cloud and your offices external IP gets blocked then thats a bigger issue.
Not sure if an RBL type of situation makes sense, but if we share the list of attackers, that would potentially be another level of prevention.
I think this same logic applies for ssh, I restrict ssh on my pbx’s, but this could be used on systems that need ssh to be available.
I think there may be value in adding a timestamp to the log file, it may have even more value if we combine our findings together.
At this stage whitelisting is supported in script via this method:
Allowed IPs can be set up in /root/distributed_BFA_Allowed_IPs.cfg(list, one per line) or any host properly listed in: /etc/hosts
Although it’s a great idea to see what is whitelisted in FPBX, would you know where it’s stored in MySQL, so I can add that logic. You are correct predicting that an IP can be blocked in described situation.
2. Sorry, what is RBL? In my environment individual lists are processed with central automation to find repeated attacker IP, that’s is blocked before entering environment on central firewall. You may do similar.
3. Same idea can go for ssh, although not a scope for the current script.
4. Yeah, for that it’ll require a bit more logic put into the script, I’ll see how quick and easy it can be added.
Thanks for the info.
I will utilize the whitelist you provided.
RBL is “Realtime Blackhole list” which is basically a list of “Bad Actors”.
These lists are heavily utilized for fighting spam within email servers.
These RBL’s are shared, so if we built a list from all contributing members of the FreePBX community, we could block a greater amount of these bad actors without ever having them touch our own server.
It is similar to what I believe you are doing where you sending to your central firewall.
The only thing is, for environments where you are on cloud, you don’t always have the ability to centralize, so an RBL would be a good method to utilize to accomplish the same thing.
I installed this on a handful of servers and will report back what I’m seeing once I collect enough data.
This gives back a lot of junk…
I would say if you want to consider it a report, then maybe that could be sent with each set of bad guys found, and active whitelists.
After running for a few days on 5 or 6 cloud based boxes, I do see a reduction in our email notifications from F2B, but still got 20,800 fail2ban, bans.
I think that number would probably be triple.
In respect to your code, I’m running it every 10 minutes and getting either nothing, 2-3 or bunches some as big as 600+.
Some will expect this from me but I will reiterate
Don’t listen to UDP/5060, it’s that easy
BUT, if you must, please understand iptables with hundreds of thousand of lines will use lots of resources and slow down your network, ipset is the solution here. If you want a blacklist, then ban the network not the host by any rule
whois -h whois .cymru.com ‘-v -f 45.254.253.9’
exposes PHMGMT-AS1 with an ASN of 22363 iun the US
whois -h whois .cymru.com ‘-v -f 45.246.107’
similary exposes the same company in HK
so although the list shows multiple hosts in various countries, the underlying culprit is the same ASN.
So having derived a blacklist of hundreds of thousands of hosts, good husbandry could reduce that , moving to an ipset again reduce resources needed.
So, not listening on UDP/5060 makes the whole concept unnecessary, but not to ignore that a templated VOIP system will have a fingerprint that leads cleverer black-hats to look for flaws outside SIP connections, check correlations to ports opened by related ip’s before SIP attacks are noticed.
This is just a small honey pot I have setup for this. While the 51XX’s generally have a low packet count it does show one thing, attempts where made. Notice how 5160, the alternate SIP port FreePBX started using about 5 years ago, is the largest hit port after 5060. Again, it doesn’t matter that the count is low because it shows attempts where made.
So if you’re going to use something not 5060, don’t just consider it the full solution because they are scanning ports. They have more resources and tools now than 10+ years ago.