OK our system got hacked and it was our fault.
We are using FreePBX Distro 10.13.66-16 and I am pretty sure they got into the system because we assigned FreePBX admin privileges to a user in UCP and then others users “inherited” the setting.
How is that possible you ask? Well apparently user management settings are “inherited” from the last record so once someone is granted Admin privilege, the next user also gets admin rights etc… etc… etc… And there there seems to be NO way to know who has been assigned admin rights because in user management it only says inherited and it does not show WHAT was inherited.
Now I need to find if other users were assigned admin rights. Is there any way to check?
Also can someone post a list of passwords that need to be changed on a compromised system?
e.g. FreePBX GUI, root, etc…
Should I change the Asterisk manager password in advanced settings?
Any other passwords?
Thanks guys. Hopefully others can learn from my mistake.
Can you possibly explain what makes you think it was hacked, and how they got into your system?
Security is my thing, so if your machine was hacked and you were running the latest version of everything, then it’s something I’m going to need to care about.
At least, can you paste the output of all your module versions? The output of fwconsole ma list would be perfect.
I know my machine was hacked because I am paying the price. I don’t know how they got in but one of the open areas were 2 UCP users with admin rights who could login to the GUI and should not have been able to do so. The rights were inherited and the Group is set not to allow PBX admin login. We are still working on cleaning the system so please provide a list of passwords that need to be changed.
I still think there should be a way to know which UCP user has admin privileges after they have been assigned and if anyone knows where to look please let me know.
fconsole ma doesn’t do anything except provide a list of commands
I’m assuming it’s pretty late where you are, but I did provide a command for you to type in, and it’s even in a different font to make it more obvious. Please take a breath, slow down, and follow instructions.
Please calm down and let people trying to help by providing what they asked for
Secondly maybe you should review the wiki but inherited means inherit from group the permission. If your group that the user belongs to has no permissions for FreePBX admin then that user can NOT login to FreePBX admin. Where are you seeing a username login? I assume some log is telling you the user. Please paste it here.
I am calm. Just tired. Time to go to sleep I guess.
I don’t know for a fact that they got in that way and I just wanted to warn everyone about this. My guess is the default group had admin privileges (we must have done that) when we created the system and a couple of users inherited it before we changed the settings. I just don’t know which users and would like to fix our mistake by finding those users and removing the GUI login capability.
184.144.169.181 - - [13/Oct/2016:18:14:25 -0400] “GET /assets/css/compiled/main/lessphp_e7ff84b53779bbf39d3fb5dbda4f7a6c241a7d3d.css HTTP/1.1” 200 41878 “http://someURL.com:81/?display=dashboard&mod=home” "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Safari/602.1.50"
184.144.169.181 - - [13/Oct/2016:18:14:24 -0400] “GET /?display=dashboard&mod=home HTTP/1.1” 200 9763 “-” "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Safari/602.1.50"
184.144.169.181 - - [13/Oct/2016:18:14:25 -0400] “GET /assets/css/compiled/main/lessphp_b67f9f3362c5d3a9341cdec44d870d6584cf6ac7.css HTTP/1.1” 200 5490 “http://someURL.com:81/?display=dashboard&mod=home” "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Safari/602.1.50"
184.144.169.181 - - [13/Oct/2016:18:14:26 -0400] “POST /index.php HTTP/1.1” 200 87 “http://someURL.com:81/?display=dashboard&mod=home” "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Safari/602.1.50"
184.144.169.181 - - [13/Oct/2016:18:14:26 -0400] “POST /index.php HTTP/1.1” 200 172 “http://someURL.com:81/?display=dashboard&mod=home” "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Safari/602.1.50"
184.144.169.181 - - [13/Oct/2016:18:14:29 -0400] “GET /?display=dashboard&mod=cdr&sub=5149999999&pjax=%23dashboard-content HTTP/1.1" 200 1558 “http://someURL.com:81/?display=dashboard&mod=home” "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Safari/602.1.50"
184.144.169.181 - - [13/Oct/2016:18:14:29 -0400] "GET /index.php?quietmode=1&module=cdr&command=grid&extension=5149999999&sort=timestamp&order=desc&limit=10&offset=0&=1476396865662 HTTP/1.1” 200 8492 “http://someURL.com:81/?display=dashboard&mod=cdr&sub=5149999999” "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Safari/602.1.50"
184.144.169.181 - - [13/Oct/2016:18:14:31 -0400] “POST /index.php HTTP/1.1” 200 172 “http://someURL.com:81/?display=dashboard&mod=cdr&sub=5149999999” "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Safari/602.1.50"
184.144.169.181 - - [13/Oct/2016:18:14:36 -0400] “POST /index.php HTTP/1.1” 200 172 “http://someURL.com:81/?display=dashboard&mod=cdr&sub=5149999999” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Safari/602.1.50”
That’s not how it works. When you removed it then it’s removed for all. It’s not set forever. If the user has inherit and you removed the option from the group then it’s gone.
184.144.169.181 - - [13/Oct/2016:18:14:36 -0400] "POST /index.php HTTP/1.1" 200 172 "http://someURL.com:81/?display=dashboard&mod=cdr&sub=5149999999" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Safari/602.1.50"
There’s nothing in these logs that look out of the ordinary. Again, what makes you think you were hacked? And what makes you think that the logs you posted above are malicious?
@tm1000. Thanks for clarifying. is it possible to have a user who does not belong to a group?
@xrobau. Call volume went up like crazy. We found the hacker files on our system. I can post it tomorrow or maybe send it to you on a private channel. It’s pretty amazing what it can do once it’s on a system.
We just saw a lot of activity in the logs to 2 or 3 extensions.
The extension account(s) themselves might have gotten hacked. How strong are the extension secrets of the extensions in question (showing in the logs)? Were they user/admin generated, or generated by FreePBX “Add Extension” function? If someone hacks an extension because of a, say, a weak password, they can initiate all sorts of unwanted calls (outbound). The other thing is go back in your HTTPD logs to dates+times that correlate to the timestamps on the hacked files. We’ve have some hacks on PHP files in the past. It could be a security vulnerability that FreePBX has not yet found or been alerted to. We got bit recently by the vulnerability in the Hotel Wakeup module (for this a fix update was also recently provided by Sangoma). If you can correlate the logs and timestamps and/or log references that access the hacked files in question, you may be able to isolate what the intruders initially used to gain hacked access. In our case, the intruders setup a spam email node. We got blacklisted and that tipped us off. After tracing it back to the FreePBX server, we discovered how they got in.
[root@ABC asterisk]# fwconsole ma refreshsignatures
Getting Data from Online Server…
Done
Checking Signatures of Modules…
Checking accountcodepreserve…
Good
Checking announcement…
Good
Checking asterisk-cli…
Good
Checking backup…
Good
Checking blacklist…
Good
Checking builtin…
Good
Checking callback…
Good
Checking callforward…
Good
Checking callrecording…
Good
Checking callwaiting…
Good
Checking cdr…
Good
Checking cel…
Good
Checking certman…
Good
Checking conferences…
Good
Checking configedit…
Good
Checking core…
Good
Checking customappsreg…
Good
Checking dashboard…
Good
Checking directory…
Good
Checking disa…
Good
Checking donotdisturb…
Good
Checking extcfg…
Signature Invalid
Could not find signed module on remote server!
Checking extensionroutes…
Good
Checking fax…
Good
Checking faxpro…
Good
Checking featurecodeadmin…
Good
Checking findmefollow…
Good
Checking framework…
Good
Checking fw_langpacks…
Good
Checking hotelwakeup…
Good
Checking iaxsettings…
Good
Checking infoservices…
Good
Checking ivr…
Good
Checking languages…
Good
Checking logfiles…
Good
Checking manager…
Good
Checking miscapps…
Good
Checking miscdests…
Good
Checking music…
Good
Checking paging…
Good
Checking pbdirectory…
Good
Checking phonebook…
Good
Checking pinsets…
Good
Checking queueprio…
Good
Checking queues…
Good
Checking recordings…
Good
Checking ringgroups…
Good
Checking sipsettings…
Good
Checking soundlang…
Good
Checking speeddial…
Good
Checking superfecta…
Good
Checking sysadmin…
Good
Checking timeconditions…
Good
Checking ucp…
Good
Checking userman…
Good
Checking voicemail…
Good
Done
Updating Hooks…Done
@xrobau. extcfg fails but that was expected since it’s maintained by us and not signed but I don’t believe they got in that way. None of the logs point that way.
@jobsoftinc extensions secrets are autogenerated (20 characters?) and we don’t know yet how they got in initially. We may never know in the end. The logs only go back a couple of weeks and the system may have gotten compromised way before that with no activity until now.
Anyways I answered all questions to the best of my knowledge. Can someone now please answer my question with regards to which passcodes we must change on the system? We already changed root and FreePBX GUI. Do we have to change the Asterisk manager password or anything else? I’d rather change one passcode too many than miss one. Thanks.
Well if you do not have AMI or MySQL opened to outside world nothing to change their,
I would still focus on your access logs in apahe to see where they came in. In that log it will show IP addresses. Look for one that does not match a subnet you trust
Also you never stated if this box has been fully updated when releases come out of if you updated after the hack. Also what ports do you have opened to the outside world as I assume you have a firewall here and no open ports to the outside world I would hope for SSH or FreePBX admin.
Most passwords are encrypted. Unencrypted passwords include SIP/IAX secrets (extensions and trunks), voicemail pins, pinsets (if applicable), that’s all I can think of.