Check to see if your /etc/freepbx.conf file has been wiped. Restoring that will likely fix your PHP issues. However, since your system was infected, you’ll likely want to restore from a system backup or wipe/reinstall the OS and FreePBX and then restore from a FreePBX backup.
Are we sure taking a backup of an infected instance and applying it over to a fresh one won’t copy some infected files over ?
In my not-so-expert opinion, as long as long as you’re restoring to a ‘clean’ system (one never compromised) and the backup just contains default module backups (and not other system files) you should be safe since modified modules should display the signature warning. Of course, it would be wise to consider the data within the modules as compromised (ie. extension and trunk passwords).
I pulled freepbx.conf from a backup and dropped it on the system – at least now things appear to be working. I’m going to keep the system offline, and basically do a new 17 install from an older backup. Keeping an eye on all our client production systems.
Other than that .clean.sh, are there other indicators to look for during our daily automated system checks?
fwconsole ma downloadinstall endpoint --edge
fwconsole ma downloadinstall endpoint --tag 17.0.2.31
This does not work for servers that don’t have an active subscription.
Confirmed just getting an xml empty error. @penguinpbx can you comment?
Additional IOCs Found - sharing for the community.
“/var/www/html/cache/monitor.php”,
“/var/www/html/cache/backend.php”,
“php -S 127.0.0.1:32764”,
“/var/www/html/.clean.sh”,
“/dev/shm/.conf_”,
“systemctl enable cleanup-helper.service”,
“systemctl enable --now systemd-udevd-helper”
195.123.233.116
Cleanup script deleted majority of it. You’ll need to find from logs/SIEM.
1 Like
Let’s clarify that statement by including that an active subscription means EPM is licensed but doesn’t have an active support renewal on it. Purchasing the annual support solves the issue.
For those that are using the free version of EPM (only for Sangoma devices) or have never licensed the EPM, the update works just fine.
Again, only expired support of an existing paid license will stop the update.
So you’re saying this does nothing for my over year old installs that are properly secured and not infected? Or do you actually mean it’s not a cure for existing compromised systems?
One would assume this closes the RCE that caused the compromise so the update should work just fine on existing systems that haven’t been compromised.
Don’t hold your breath.
Sorry if I missed it, but it is not clear to me:
- How to identify if a system has been compromised or not.
- How to deal with compromised PBX systems. Do I need to wipe and recreate from scratch?
These 2 points should be 100% visible when one enters this thread.
2 Likes
Somehow the package also doubled in size… Whatever broke now is fixed with 100% more code lol
2 Likes
Three of the biggest indicators:
-
You have .clean.sh in the base html directory:
/var/www/html/.clean.sh -
Your freepbx.conf file has been cleared out or modified.
The link to the other thread has some steps to check for compromises.
1 Like
Same. I get that error if I run:
fwconsole ma downloadinstall endpoint --edge
Downloading module 'endpoint'
The following error(s) occured:
- Retrieved Module XML Was Empty
If I run
fwconsole ma downloadinstall endpoint --tag 17.0.2.31
Unable to update module endpoint - 17.0.2.31, it does not exist:
Can someone at Sangoma please advise ASAP.
Upgrade your repository.
fwconsole ma refreshsignatures
fwconsole ma downloadinstall endpoint --tag 17.0.2.31
I get exactly the same error in my previous post. All of the signatures were fine.
Did you pay for EPM at any point?
This might be a silly question, but why wouldn’t a new FreePBX install by default lock down access to the ACP to only trusted networks that are defined? Say the deployment’s LAN subnet is an initial default. And the customer can then go in to define additional trusted networks to the list. I haven’t done a new install in years, so maybe this is already the case and I’m speaking out of turn.
You look at a hardware firewall. Typically by default a new deployment blocks all inbound access on the outside port. Then the customer can go into the config to allow specified traffic through the outside port. Or a cloud-based resource like on Azure. By default the networking whitelist for inbound access is usually empty, with the customer’s IP endpoint appearing in a prompt to be added to the whitelist.
It does unless you tell it not to (or don’t activate the firewall). I think this is a very important data point that people need to keep in mind. You cannot control the choices of the individual users. In the other thread people openly admit the had the admin GUI exposed to the Internet in some unsafe form.
It really doesn’t matter how many firewalls are in front of the system if someone went in and exposes the system GUI access in all the firewalls. Unfortunately we can’t write modules to stop that.
Yes there was a RCE in the code, honestly there probably is more that haven’t been found yet, but at the same time the human factor (i.e. people turning off or having poorly configured security) is an outside factor that Sangoma can’t account for.
1 Like