I’ve worked on quite a few compromised servers before. If you would be willing to let me have a copy of the VM image I’d be very interested in have a look, to see if I could find any clues as to how it was compromised.
I’m not aware there’s any proof that this is related to the Apache/PHP upgrade.
Obviously the server image would be treated in strictest confidence.
If you are willing please drop me a line using the form here - http://sysadminman.net/contact.php
Matt
Moderator note: No need to go off to other site, we will keep the information available here. Thanks.
Here’s a summary of what we currently know about this vulnerability:
Vulnerability: Any stock CentOS 5.x or 6.x system with Apache and PHP exposed to Internet access with no IP address restrictions (WhiteList).
How Do They Get In: Some (perhaps unknown) vulnerability in stock versions of Apache and PHP on CentOS systems allows the attacker to gain system access. We really don’t know any more than that at this juncture. But this does not appear to be a PHPmyAdmin exploit as that utility is locked down by secure htaccess password on some systems that have been compromised. Fail2Ban is not detecting hack attempts so it appears the attacker is walking right in with this exploit.
Privileges: Still unclear whether attacker is gaining root access or merely same access as enjoyed by Apache on the attacked system. To do what they’re doing would NOT require root privileges on your system. The attacker brings a customized version of WebMin with their own password.
What Happens Once They’re In: In a nutshell, your system is turned into a zombie. Using perl and WebMin (their own version), they can interconnect your server into a worldwide network of machines used to launch denial-of-service and other malicious attacks against other systems on the Internet.
How Do I Know If My Machine Has Been Compromised? Examine some of the previous comments in this thread. Run ps awx on your server and look for long lists of processes running perl scripts. Look in the /usr directory for a directory called game, games, books, etc. Inside those directories, run ls -all which will show hidden files beginning with a period. There will be a directory called .n or .s or something similar. Look in /etc/cron.daily. There will be a new script as outlined in this thread. NOTE: The zombie software is old and signatures already exist in anti-virus programs. The exploit to gain access may be entirely new.
What Should Be in /usr? On a stock FreePBX Distro system, you should see the following directories:
bin etc games include java kerberos lib libexec local sbin share src tmp X11R6
The games directory will be empty when you ls -all
What Should Be in /etc/cron.daily? On a stock FreePBX Distro system, you should see the following files:
0anacron 0logwatch cups logrotate makewhatis.cron mlocate.cron prelink rpm tmpwatch
How to Fix It: If your system has been compromised, reformat the disk and reinstall. If they haven’t gotten in or if you’ve started over, (1) immediately turn off Internet access to web services on your servers. You can implement a whitelist of safe IP addresses for web access using IPtables or your hardware-based firewall. (2) Upgrade using out script found here
NOTE: Just because your system has not yet been compromised does NOT mean you are safe. Your system still needs to be secured. Turn OFF Web Access Now!
This information only relates the the OPs post. He was kind enough to let me have a copy of the hacked servers drive image. This may or may not relate to any other hacking cases.
I’m 99% sure that the exploit used a vulnerability in phpmyadmin.
The exploit is similar to the the one included here - http://sourceforge.net/tracker/?func=detail&aid=3045132&group_id=23067&atid=377408, although the exact code there does not work (at least I couldn’t get it to work). The version of phpmyadmin on the hacked server was 2.11.9.6-1
If you have this file on your server I recommend deleting, or making it inaccessible ASAP - /usr/share/phpmyadmin/scripts/setup.php. The hackers had deleted this file on the hacked box as part of the hack.
I will post more information and the exact exploit when I find it.