Our freePBX server was hacked this weekend and used to run up a big bill of international calls. I was alerted by our VoIP provider. I am puzzled because the box was protected by a hardware firewall, a Juniper J series router. It wouldn’t have been possible to SSH to the box or register an extension. However, port 80 was open to the outside world, I didn’t have default username and password set for the GUI.
There is no record of the calls in the GUI but I can see them in /var/log/asterisk/full
We nipped this in the bud thanks to precautions our VoIP provider have in place but as we are a small enough company this could have done a lot of financial damage.
Can this type of attack take place on Port80 and if so how?
I have changed the default ports and the GUI is no longer reachable from the public internet.
I would appreciate any tips or links to relevant articles.
Obviously untrusted access to the Admin GUI is not recommended, as there have been exploits in the past that only required access to the Admin GUI page without login. I know of no current exploits tho. Without ssh access and without calls showing up in the CDR, my suspicion is that you are a victim of some variation of “thanku”. Check for include files of non-zero filesize:
[[email protected] ~]# ll /etc/asterisk/ | egrep "custom|post"
-rw-rw-r--. 1 asterisk asterisk 0 Aug 1 07:30 ari_additional_custom.conf
-rw-rw-r--. 1 asterisk asterisk 0 Aug 1 07:30 ari_general_custom.conf
-rw-rw-r--. 1 asterisk asterisk 0 May 15 10:52 ccss_general_custom.conf
You are not the only one reporting this. I had a few people contact me to ask if I could look at their box to figure out how they were compromised. The one box I got into had the apache logs wiped out. The asterisk logs did not show any brute force on registering an end point. The hacker either found a vulnerability somewhere in FreePBX or somehow got phone configs that show the SIP secrets.
How are you provisioning your phones? Meaning are you using TFTP or HTTP(S) and are you using username and password to protect the provisioning?
If extension SIP secrets get leaked allowing spurious registrations, then the malicious calls would show up in the CDR.
Thanks for the reply. I’m certain there is no way a hacker could register as the only port open to the public internet was port 80. For this sever we just use SIP clients on laptops. We have another server and the MTAs are provisioned via TFTP. This server was not compromised.
As I said I can see the calls in /var/log/asterisk/full no other trace of the compromise. Fail2ban - nothing.
As soon as our peer blocked the trunk they vanished leaving little trace.
I’ve tested my firewall rules today and I can’t register an extension from outside our network.
Yes and I see no extensions registering in /var/log/asterisk/full before the calls were made. These were one way audio calls, our peer had recordings. It is possible to install a dialing script or similar via http? The GUI gave access to the asterisk CLI module. I’m no longer using this server it’s been taken offline. I wouldn’t trust it again, and the server I spun up to replace it is not accessible form the public internet.
IMO this should be investigated with some priority.
Are the fraudulently called numbers the same or similar, e.g. all in Palestine?
What do the victims have in common (FreePBX and Asterisk version, ports open, etc.)?
If not obliterated by the attacker, what do the Apache logs show?
In /var/log/asterisk/full, how does it show the calls being made (from an existing extension, from a newly created extension, some other way)?
Does the log show a CDR entry being made? If yes, any clues as to how it got wiped? If no, is that normal for the way the call was made, or is there evidence that the dial plan was modified?
Comparing files in /etc/asterisk with a recent backup, are there any unexplained changes or additions? Similarly for /var/www?
Note that registration is not necessary in order to place calls. A SIP thief would be more likely to use the stolen auth credentials to place calls through your PBX without registering, leaving a smaller footprint.
[Edited to remain on subject - mod]
Are you sure? do you have a SIP Trunk, how does it connect?
Can you post a log from one of these calls?
(Keep in mind that logs are usually saved for only 7 days so look for calls within the last week)
Just change the local IP so any port forwarding rules won’t work, and disable the trunk. Once that’s done there’s no way an external device can place calls on your PBX.
This thread is IMPORTANT. All posts not directly related to the subject WILL BE DELETED.
Just a fly on the wall here, but I would definitely be interested in hearing the outcome of how the security loophole is found and patched. Running FreePBX 13 here. I have the server on the LAN, with no public Internet exposure. But if some exploit vector exists merely via the web interface’s HTTP port 80 that’s definitely scary for some.
Here’s something I found while searching out there. Assuming since this a few years old it’s been patched, right?
One of the first thing I would check is if tcp/5038 is open to the interwebs, if the GUI has been compromised and manager details leaked, an indirect approach would be less detectable and relatively easy to accomplish. Same would go for any restful thingies on less protected ports
Well this one would be a concern I think:
Race condition in the mod_status module in the Apache HTTP Server before 2.4.10 allows remote attackers to cause a denial of service (heap-based buffer overflow), or possibly obtain sensitive credential information or execute arbitrary code, via a crafted request that triggers improper scoreboard handling within the status_handler function in modules/generators/mod_status.c and the lua_ap_scoreboard_worker function in modules/lua/lua_request.c.
@ChrisMaverley Chris, can you please provide the information we asked you? that would help determine how the calls were originated.
- The Customs conf files.
- A call log. (You can use pastebin as mentioned in the wiki link above)
No, not by obfuscation, by proactive action, so you don’t use 5060 but decide to drop anyone that tries, (exception for your trunks needed), don’t just drop them, drop their entire underlying network, very shortly you will be “silent as the lamb” when it comes to SIp atttacks. If you have a working portscanner on your firewall, then that gets you brownie point.
Over the years, I have been watching this stuff, this will have to be anecdotal because anything I found is long unnecessary and has been discarded.
These guys are not “knuckle-draggers”, they are very sophisticated and often funded by the States of China, Palstine and Russia, almost all are cleverer than at least me.
They decide who to vector by many metrics, they work in co-ordination, one set will probe the fingerprint of your outside IP, looking for 5060, 5038, the set of provisioning ports, they will also try looking for html scripts that although not necessary vulnerable, identify a system that is open for further probing. Look to implementing Fail2Ban’s apache noscripts jail, its already there just needs turning on
Every few weeks/months another profile is developed and used.
These low-level dudes , glean info and escalate your identity upwards to more powerful groups if you fail the low level scripts.
(Just because it says its sipvicious , it’s not necessarily so )
Two radical but effective methods I find useful:-
- Just don’t use 5060.
- Only accept traffic to your domain name and reject anything to your ip address (yes, you need a domain name, and a SSL certificate)
^^^ That is not something that is possible. A Domain name is just a lookup record. When you say ‘Connect to xrobau.com’, what your computer does is say ‘Hey, DNS, what’s the address of xrobau.com?’ - DNS says ‘xrobau.com is at 188.8.131.52’ and your computer then connects to 184.108.40.206.
What @dicko MIGHT be trying to say is ‘only allow connections via https’, which doesn’t really achieve anything, apart from stopping man in the middle attacks, and co-incidentally, making it easier to figure out the hostname of a machine.
For example, let’s pick a random machine on the internet, 220.127.116.11 (which just happens to be mirror.freepbx.org) and try to figure out what it is:
[[email protected] ~]# openssl s_client -connect 18.104.22.168:443 < /dev/null 2>/dev/null | openssl x509 -noout -text | grep CN=
Issuer: C=US, ST=Arizona, L=Scottsdale, O=GoDaddy.com, Inc., OU=http://certs.godaddy.com/repository/, CN=Go Daddy Secure Certificate Authority - G2
Subject: OU=Domain Control Validated, CN=*.freepbx.org
[[email protected] ~]#
That’s saying the certificate is ‘*.freepbx.org’. From an IP address, I’ve learned that it has something to do with freepbx.org. I could then look at apache headers, etc.
The only way to properly secure a machine is with a real firewall - and noticing that fortigate was mentioned earlier in this thread - fortigate is NOT a real firewall.
It was not mentioned until you did. But in case it’s of value to OP, I know of several PBXs successfully hacked for toll fraud over the last week, all of which were 100% due to a fortigate exploit.
And again, please everyone, keep to the topic. More general discussions on security arising from here can be continued in new threads.
I’m the one hiding off topic posts! And everyone was warned twice now. This is not the place for discussion about what should have been done.
I had questioned if the circa-2016 FreePBX exploits were patched since being made public. Still wondering if my assumption/hope is correct they had been. Googled this working example, which pretty much aligns with what the OP observed. If their only exposed port was HTTP, then this scenario could occur if the exploit vector is still present, right?
OP, if you look in your /var/www/html directory you might see some questionable PHP files with newer dates I’m thinking. If so, these serve as the payload for what the exploiting party dropped off…