We have a self-hosted FreePBX server (Currently on version 14.0.13.28)
Today our FreePBX system apparently made a few hundred outgoing calls to the Dominican Republic, there are no logs of anyone SSHing into our server, and our web portal is behind a firewall only allowing specific traffic to access it.
We utilize SIP trunks, hooked up with Twilio for our inbound and outbound calls.
Immediately after this happened, the server crashed, and upon reboot I ran the fpbx updates. This appears to have stopped for the time being, but I’m worried that the issues came from an exploit or hack that has permanently opened our FreePBX server in a way I’m not familiar with.
Are there areas I should check in the FreePBX server for why this would have happened? Is there anything other than updating the OS and modules that I should be doing past this point? How likely was this just an exploit on a vulnerability that was patched with my updates? (I believe the last time I updated the modules on the server was 3-4 months prior).
The most common vector for such an attack is for a SIP secret to leak or a successful brute force against a SIP account. Once they have the SIP secret, they can generate outbound calls. The other possibility is that your GUI was running an old version of framework from last year and was exposed to untrusted traffic, which you say is not the case.
You can search for spurious registrations with the following greps:
And just to re-shout what has been shouted many times before, 99.999% of these penetrations are against, UDP:5060 (and now 5160).
It is amazingly trivial for almost everyone to just “not do that” for your external extensions, and trunks just don’t need that ip:port pair to be widely open as they are specific to the VSP
I wish there are security monitoring / auditing tools (such as Tripwire, Logwatch, Rootkit Hunter, or a build in script like this or this or something else) implemented in FreePBX. Hacking and toll fraud will only be an increasing threat to the PBX community and I have seen calls for help frequently here.
I think security monitoring / auditing tools is a highly needed feature for FreePBX and hopefully not difficult to integrate as the tools and skills are readily available.
It is trivial to add all the above, personally I install both logwatch and RKH on initial install, RKH or its lookalikes MUST be installed before deploying the PBX or they can’t be trusted for obvious reasons, again, what is hard for anybody to change the listening port of both chan_sip and chan_pjsip, just do it, then think about using use TCP over UDP (easy) and TLS over TCP ( quite easy) , then show me ANY successful (or unsuccessful for that matter) connections outside 5000-5999 .
JM2CWAE, but I have being doing all this for quite some years after losing several thousand dollars to my own lack of knowledge, since then I make it a project to set up honeypots and watch the evolving attacks. A naked un-customized native FreePBX in my opinion displays all the fingerprints that invite “secondary attacks” just like having ssh on 22 or rdp on 3389.
NAT’ing. Using an alternate port requires planning. Not all FreePBX users (me included) have the skills or experience to do so. I recently saw a post where OP seems to have issues with non standard ports as well.
If you can write something in the FreePBX Documentation Center will be nice. Sangoma Documentation
Natting is a red herring, if you need to change the iptable forward rule to suite that change should be obvious. There are two lines you need to set in your ‘sip settings’ one for chan_sip and one for chan_pjsip to change your ‘listening port’, then when you deploy the phones they just need to be aware of the port they are expected to signal on, it really is that simple.
Allowing TCP over UDP, needs another two or three adjustments, most every phone can use TCP instead of UDP, that will reduce your profile to ‘very small’, going to TLS further will well secure you from all this crap.
I was able to secure my pbx by updating fail2ban on the followinf config file: /etc/fail2ban/filter.d $ cat asterisk.conf.
Adding following conditions:
^NOTICE.* .*: Registration from ‘.*’ failed for ‘<HOST>’ – Wrong password
^NOTICE.* .*: Registration from ‘.*’ failed for ‘<HOST>:.*’ – No matching peer found
^NOTICE.* .*: Registration from ‘.*’ failed for ‘<HOST>’ – No matching peer found
^NOTICE.* .*: Registration from ‘.*’ failed for ‘<HOST>’ – Username/auth name mismatch
^NOTICE.* .*: Registration from ‘.*’ failed for ‘<HOST>’ – Device does not match ACL
^NOTICE.* .*: Registration from ‘.*’ failed for ‘<HOST>’ – Peer is not supposed to register
^NOTICE.* .*: Registration from ‘.*’ failed for ‘<HOST>’ – ACL error (permit/deny)
^NOTICE.* .*: Registration from ‘.*’ failed for ‘<HOST>’ – Device does not match ACL
^NOTICE.* <HOST> failed to authenticate as ‘.*’$
^NOTICE.* .*: No registration for peer ‘.*’ \(from <HOST>\)
^NOTICE.* .*: Host <HOST> failed MD5 authentication for ‘.*’ (.*)
^NOTICE.* .*: Failed to authenticate user .*@<HOST>;.*
^NOTICE.* .*: Sending fake auth rejection for device .*\<sip:.*\@<HOST>\>;tag=.*
added under default condition as per below… also updated my iptables to drop traffic from ip’s that kept on hammering the pbx with wrong password…
failregex = ^Registration from ‘[^’]’ failed for ‘(:\d+)?’ - (?:Wrong password|Username/auth name mismatch|No matching peer found|Not a local domain|Device does not match ACL|Peer is not supposed to register|ACL error (permit/deny)|Not a local domain)$
^Call from ‘[^’]’ (:\d+) to extension ‘[^’]’ rejected because extension not found in context
^(?:Host )? (?:failed (?:to authenticate\b|MD5 authentication\b)|tried to authenticate with nonexistent user\b)
^No registration for peer ‘[^’]’ (from )$
^hacking attempt detected ‘’$
^SecurityEvent="(?:FailedACL|InvalidAccountID|ChallengeResponseFailed|InvalidPassword)"(?:(?:,(?!RemoteAddress=)\w+="[^"]")|.?),RemoteAddress=“IPV[46]/[^/”]+//\d+"(?:,(?!RemoteAddress=)\w+="[^"]")$
^“Rejecting unknown SIP connection from (?::\d+)?”$
^Request (?:’[^’]’ )?from ‘(?:[^’]|.?)’ failed for ‘(?::\d+)?’\s(callid: [^)]) - (?:No matching endpoint found|Not match Endpoint(?: Contact)? ACL|(?:Failed|Error) to authenticate)\s$
^NOTICE.* .: Registration from ‘.’ failed for ‘’ – Wrong password
^NOTICE.* .: Registration from ‘.’ failed for ‘:.’ – No matching peer found
^NOTICE. .: Registration from ‘.’ failed for ‘’ – No matching peer found
^NOTICE.* .: Registration from ‘.’ failed for ‘’ – Username/auth name mismatch
^NOTICE.* .: Registration from ‘.’ failed for ‘’ – Device does not match ACL
^NOTICE.* .: Registration from ‘.’ failed for ‘’ – Peer is not supposed to register
^NOTICE.* .: Registration from ‘.’ failed for ‘’ – ACL error (permit/deny)
^NOTICE.* .: Registration from ‘.’ failed for ‘’ – Device does not match ACL
^NOTICE.* failed to authenticate as ‘.’$
^NOTICE. .: No registration for peer ‘.’ (from )
^NOTICE.* .: Host failed MD5 authentication for ‘.’ (.)
^NOTICE. .: Failed to authenticate user .@;.*
^NOTICE.* .: Sending fake auth rejection for device .<sip:.@>;tag=.
Following are hacker ip’s as per various websites and actually observed hammering my pbx…
iptables -I INPUT -s 185.53.88.0/24 -j DROP
iptables -I INPUT -s 134.122.23.0/24 -j DROP
iptables -I INPUT -s 45.0.0.0/8 -j DROP
iptables -I INPUT -s 51.0.0.0/8 -j DROP
iptables -I INPUT -s 107.167.0.0/16 -j DROP
iptables -I INPUT -s 188.161.0.0/16 -j DROP
iptables -I INPUT -s 95.217.0.0/16 -j DROP
iptables -I INPUT -s 193.202.0.0/16 -j DROP
iptables -I INPUT -s 212.129.0.0/16 -j DROP
iptables -I INPUT -s 80.211.0.0/16 -j DROP
iptables -I INPUT -s 158.140.0.0/16 -j DROP
iptables -I INPUT -s 185.97.0.0/16 -j DROP
iptables -I INPUT -s 77.247.0.0/16 -j DROP
iptables -I INPUT -s 37.49.0.0/16 -j DROP
iptables -I INPUT -s 202.21.0.0/16 -j DROP
iptables -I INPUT -s 45.143.0.0/16 -j DROP
iptables -I INPUT -s 142.44.0.0/16 -j DROP
iptables -I INPUT -s 209.107.0.0/16 -j DROP
iptables -I INPUT -s 104.238.0.0/16 -j DROP
iptables -I INPUT -s 62.210.0.0/16 -j DROP
iptables -I INPUT -s 52.170.0.0/16 -j DROP
iptables -I INPUT -s 194.55.0.0/16 -j DROP
iptables -I INPUT -s 45.10.0.0/16 -j DROP
iptables -I INPUT -s 45.56.0.0/16 -j DROP
iptables -I INPUT -s 209.234.0.0/16 -j DROP
iptables -I INPUT -s 5.183.0.0/16 -j DROP
iptables -I INPUT -s 198.12.0.0/16 -j DROP
iptables -I INPUT -s 163.172.0.0/16 -j DROP
iptables -I INPUT -s 206.221.0.0/16 -j DROP
iptables -I INPUT -s 156.96.0.0/16 -j DROP
iptables -I INPUT -s 45.151.0.0/16 -j DROP
iptables -I INPUT -s 206.81.0.0/16 -j DROP
iptables -I INPUT -s 103.145.0.0/16 -j DROP
iptables -I INPUT -s 199.127.0.0/16 -j DROP
iptables -I INPUT -s 212.83.0.0/16 -j DROP
iptables -I INPUT -s 51.89.0.0/16 -j DROP
iptables -I INPUT -s 185.174.0.0/16 -j DROP
iptables -I INPUT -s 80.246.0.0/16 -j DROP
iptables -I INPUT -s 185.147.0.0/16 -j DROP
iptables -I INPUT -s 104.243.43.0/24 -j DROP
iptables -I INPUT -s 162.243.160.0/24 -j DROP
iptables -I INPUT -s 82.135.148.0/24 -j DROP
iptables -I INPUT -s 84.88.40.0/24 -j DROP
iptables -I INPUT -s 190.0.63.0/24 -j DROP
iptables -I INPUT -s 113.141.67.0/24 -j DROP
ture, this is a recurrent issue. This is why I think security monitoring / auditing tools need to be part of FreePBX and not up to individual users to install it. Hope this will provide peace of mind and save tons of money. I believe this is a highly needed feature.
It would be worth starting a new thread to discuss options. I agree with this idea and would be interested in discussing existing solutions vs. something custom built for FreePBX.
I am happy to do so. I will do some research, write something down then ask for feedback from the community. This may take some time but hopefully we will have something useful.
And hopefully @wesleys has also. if he is still getting hundreds of calls an hour after following my suggestion, I would be surprised, if that problem has gone away for him, i personally am gratified
Writing here from my own experience, I agree about setting up two safeguards:
i) Port masking/NATting as stated before. You’ll need a solid planning in order to do well routing.
ii) Just set looong, random passwords for all accounts, either extensions or trunk ones. I made a small shell-script to achieve this apart PBX portal. Fancy a good set of 64/128/256-random character passwords? (beat this damn brute-force nerds muahahah!)
"And just to re-shout what has been shouted many times before, 99.999% of these penetrations are against, UDP:5060 (and now 5160).
It is amazingly trivial for almost everyone to just “not do that” for your external extensions, and trunks just don’t need that ip:port pair to be widely open as they are specific to the VSP"
My FreePBX system is on the inside private network behind a Cisco router running translation code. I have absolutely tried using a port other than 5060 to allow for my softphone on my laptop an my cell phone to connect into the FreePBX system even including all of the special SIP port forwarding configurations that Cisco recommends. It does not work. The registration regularly drops. Only a port forward on 5060 is reliable and I attribute that to hidden “special” NAT code that is in the Cisco IOS.
Yes I could use a client VPN from the remotes and I have it setup that way but it’s an extra nuisance…
I have international dialing blocked on my trunks AT THE PHONE COMPANY and I use the longest and most random SIP passwords that my extensions will support including the softphones. And I change them regularly on the softphones just in case some smartguy wifi sniffer in a Starbucks somewhere has picked them up.
Behind any brute force attack is a weak password IMHO. I’m more than happy for the bad guys to waste years attempting to brute-force one of my much-larger-than-8-character random strong passwords on one of my Polycom extensions just to get a LD extension that they could make the same lower-48 unlimited calling on that any 2-bit throwaway cell phone can make. Good luck with that. I would bet the OP has a SIP password on one of his extensions of 111111 or something simple. Usually crackers go for the weakest and simplest point with the largest return. Nobody is going to do esoteric cracks to be able to make unlimited nationwide calls just like nobody is going to buy liquid nitrogen to crack-off a Kryptonite bicycle lock that is securing a rusty old B.S.O.
For interest to your comments dicko, I changed my listening port to 45060 and it immediately started getting hammered. Maybe my choice wasn’t too clever by just adding a 4 to the regular port…but it didn’t seem to particularly hide things from hackers.
)
ture, this is a recurrent issue. This is why I think security monitoring / auditing tools need to be part of FreePBX and not up to individual users to install it. Hope this will provide peace of mind and save tons of money. I believe this is a highly needed feature.