Hacked again :/

Drrehak · October 12, 2020, 12:41am

damnit, i’m hacked again. (FreePBX 14.0.13.40)

Freepbx seems to be super vulnerable now. I have had my second system hacked.

I do my best to keep things up to date. I dont understand how its happened again. I just logged in to the webgui and I see there are modules to be updated. I thought crucial ones would update automatically?

I am afraid as a small business, I need to switch to something like Ooma or Verizon’s business phone.

Anyone else having these problems?

dicko · October 12, 2020, 1:11am

what problems? you don’t say

Drrehak · October 12, 2020, 2:38am

I’m just frustrated that modules need to be constantly updated.

This is from my hosting provider:
“The original purpose of this hack was to allow an attacker to place long distance calls through your system. However there may also several other suspicious files in your web root that need attention as well.

The root cause was a vulnerability in a FreePBX module that most likely was not updated in time to resolve the issue.”

Drrehak · October 12, 2020, 3:42am

What do you mean?

Stewart1 · October 12, 2020, 3:48am

I’m not certain, but am fairly sure that the attacker needs access to the admin GUI to exploit the recent module vulnerabilities. Did you leave this accessible from anywhere? If not, if you were using a Let’s Encrypt certificate, you may have had a vulnerable module that left the firewall disabled after a failed renewal.

Does your hosting service provide a firewall separate from your VPS? If so, did you set it up to limit admin access?

Do you have a pre-breach backup to restore from? If so, set it up on a new VPS, secure and update it properly, confirm that it works correctly, then abandon the compromised one.

Drrehak · October 12, 2020, 1:05pm

this happened 2 months ago. 1 month ago I did a clean install. I updated passwords. Responsive firewall was active. I had an up to date system. This weekend Vitelity suspends my DID because of a $650 negative balance !!! (I cant get through to them until 10am EST)

I dont know what a Let’s Encrypt certificate is.
Is an additional firewall necessary? (this proves my point of Freepbx vulnerablility)

dicko · October 12, 2020, 1:51pm

Can you confirm that the calls originated from your FreePBX box ?

Drrehak · October 12, 2020, 1:53pm

how would I do that?

I see this asterisk activity right now! not sure if this is normal…

[2020-10-12 09:48:08] NOTICE[31776] res_pjsip/pjsip_distributor.c: Request ‘REGISTER’ from ‘“7023” <sip:7023@XXXXX>’ failed for ‘108.62.123.167:5669’ (callid: 3112690313) - Failed to authenticate
[2020-10-12 09:48:08] NOTICE[31776] res_pjsip/pjsip_distributor.c: Request ‘REGISTER’ from ‘“7023” <sip:7023@XXXX>’ failed for ‘108.62.123.167:5669’ (callid: 3697639808) - No matching endpoint found after 65 tries in 1.061 ms
[2020-10-12 09:48:08] NOTICE[31776] res_pjsip/pjsip_distributor.c: Request ‘REGISTER’ from ‘“7023” <sip:7023@XXXX>’ failed for ‘108.62.123.167:5669’ (callid: 3697639808) - Failed to authenticate
[2020-10-12 09:48:08] NOTICE[31776] res_pjsip/pjsip_distributor.c: Request ‘REGISTER’ from ‘“7023” <sip:7023@XXXXX>’ failed for ‘108.62.123.167:5669’ (callid: 2091690227) - No matching endpoint found after 66 tries in 1.066 ms
[2020-10-12 09:48:08] NOTICE[31776] res_pjsip/pjsip_distributor.c: Request ‘REGISTER’ from ‘“7023” <sip:7023@XXXXX>’ failed for ‘108.62.123.167:5669’ (callid: 2091690227) - Failed to authenticate
[2020-10-12 09:48:08] NOTICE[31776] res_pjsip/pjsip_distributor.c: Request ‘REGISTER’ from ‘“7023” <sip:7023@XXXXX>’ failed for ‘108.62.123.167:5669’ (callid: 453101510) - No matching endpoint found after 67 tries in 1.073 ms
[2020-10-12 09:48:08] NOTICE[31776] res_pjsip/pjsip_distributor.c: Request ‘REGISTER’ from ‘“7023” <sip:7023@XXXX>’ failed for ‘108.62.123.167:5669’ (callid: 453101510) - Failed to authenticate
[2020-10-12 09:48:08] NOTICE[31776] res_pjsip/pjsip_distributor.c: Request ‘REGISTER’ from ‘“7023” <sip:7023@XXXX>’ failed for ‘108.62.123.167:5669’ (callid: 2465888142) - No matching endpoint found after 68 tries in 1.076 ms
[2020-10-12 09:48:08] NOTICE[31776] res_pjsip/pjsip_distributor.c: Request ‘REGISTER’ from ‘“7023” <sip:7023@XXXXX>’ failed for ‘108.62.123.167:5669’ (callid: 2465888142) - Failed to authenticate
[2020-10-12 09:48:08] NOTICE[31776] res_pjsip/pjsip_distributor.c: Request ‘REGISTER’ from ‘“7023” <sip:7023@XXXXX>’ failed for ‘108.62.123.167:5669’ (callid: 3150506533) - No matching endpoint found after 69 tries in 1.079 ms
[2020-10-12 09:48:08] NOTICE[31776] res_pjsip/pjsip_distributor.c: Request ‘REGISTER’ from ‘“7023” <sip:7023@XXXXXX>’ failed for ‘108.62.123.167:5669’ (callid: 3150506533) - Failed to authenticate
[2020-10-12 09:48:08] NOTICE[31776] res_pjsip/pjsip_distributor.c: Request ‘REGISTER’ from ‘“7023” <sip:7023@XXXXXX>’ failed for ‘108.62.123.167:5669’ (callid: 2091690227) - No matching endpoint found after 70 tries in 1.097 ms
[2020-10-12 09:48:08] NOTICE[31776] res_pjsip/pjsip_distributor.c: Request ‘REGISTER’ from ‘“7023” <sip:7023@XXXXXX>’ failed for ‘108.62.123.167:5669’ (callid: 2091690227) - Failed to authenticate
[2020-10-12 09:48:08] NOTICE[31776] res_pjsip/pjsip_distributor.c: Request ‘REGISTER’ from ‘“7023” <sip:7023@XXXXXX>’ failed for ‘108.62.123.167:5669’ (callid: 3150506533) - No matching

dicko · October 12, 2020, 1:56pm

you need to get your fail2ban working, it would catch all that.

stopgap measure from a shell

sudo iptables -A INPUT -s 108.62.120.0/21 -j DROP

mcgrathr · October 12, 2020, 2:03pm

The point is to know where have been hacked, someone could said ja! Obliviously that’s common sense, but should take a look and analyzing logs files date when presumable system have been hacked.

Should be on message details as about "Register" and "invite" message and so on with others events.

By the way hacked it mainly happens when it use remote extension, if so better to use VPN connection or block anonymous and remote administration and so on with any vulnerable point which caused with the hacked events and check on router what ports are opened and close it (it better use a SBC when repeatedly happens hacks).

slobera · October 12, 2020, 2:23pm

I just edited your comments cause you was showing your PBX’s public ip.
BTW, it seems that your interface is configured as “Local” or “Trusted”.
Cause I was able to reach your webUI and your SSH port.
My suggestion is to open a support ticket with us or configure the firewall properly.
Also, be sure to not be running tftp or http provisioning without authentication.

Drrehak · October 12, 2020, 2:26pm

thank you! I have used freepbx flawlessly for 10 yrs. and i’ve never had to dig in like this…

Vitelity sent these CDR

10/11/2020 13:45	drre_9676	14849388269	1.8	0.0144	0.03	ANSWERED	drre_9676	sandhopper1	Dial	SIP/6214849388269@fwd4
10/11/2020 12:23	drre_9676	17017172813	72.7	0.0144	1.05	ANSWERED	drre_9676	piggy2	Dial	SIP/6217017172813@fwd5
10/11/2020 13:21	drre_9676	17017172813	14.6	0.0144	0.21	ANSWERED	drre_9676	piggy2	Dial	SIP/6217017172813@fwd5
10/11/2020 9:30	drre_9676	17017172813	245.7	0.0144	3.54	ANSWERED	drre_9676	piggy2	Dial	SIP/6217017172813@fwd5
10/11/2020 9:00	drre_9676	17017172813	275.9	0.0144	3.97	ANSWERED	drre_9676	piggy2	Dial	SIP/6217017172813@fwd5
10/11/2020 9:29	drre_9676	17017172813	245.3	0.0144	3.53	ANSWERED	drre_9676	piggy2	Dial	SIP/6017017172813@fwd17
10/11/2020 9:59	drre_9676	17017172813	216	0.0144	3.11	ANSWERED	drre_9676	piggy2	Dial	SIP/6217017172813@fwd5
10/11/2020 12:28	drre_9676	17017172813	67	0.0144	0.96	ANSWERED	drre_9676	piggy2	Dial	SIP/6217017172813@fwd5
10/11/2020 9:02	drre_9676	17017172813	272.3	0.0144	3.92	ANSWERED	drre_9676	piggy2	Dial	SIP/6017017172813@fwd17
10/11/2020 9:27	drre_9676	17017172813	247.7	0.0144	3.57	ANSWERED	drre_9676	piggy2	Dial	SIP/6217017172813@fwd5
10/11/2020 9:28	drre_9676	17017172813	247.7	0.0144	3.57	ANSWERED	drre_9676	piggy2	Dial	SIP/6217017172813@fwd5
10/11/2020 13:20	drre_9676	17017172813	15.2	0.0144	0.22	ANSWERED	drre_9676	piggy2	Dial	SIP/6217017172813@fwd5
10/11/2020 13:22	drre_9676	17017172813	13.3	0.0144	0.19	ANSWERED	drre_9676	sithandra1	Dial	SIP/6217017172813@fwd5
10/11/2020 13:20	drre_9676	17017172813	15.2	0.0144	0.22	ANSWERED	drre_9676	sithandra1	Dial	SIP/6217017172813@fwd5
10/11/2020 13:20	drre_9676	17017172813	15.2	0.0144	0.22	ANSWERED	drre_9676	sithandra1	Dial	SIP/6217017172813@fwd5
10/11/2020 8:55	drre_9676	17017172813	280.3	0.0144	4.04	ANSWERED	drre_9676	sithandra1	Dial	SIP/6217017172813@fwd5
10/11/2020 8:55	drre_9676	17017172813	280.3	0.0144	4.04	ANSWERED	drre_9676	sithandra1	Dial	SIP/6217017172813@fwd5
10/11/2020 9:24	drre_9676	17017172813	250.8	0.0144	3.61	ANSWERED	drre_9676	sithandra1	Dial	SIP/6217017172813@fwd5
10/11/2020 12:23	drre_9676	17017172813	72.6	0.0144	1.05	ANSWERED	drre_9676	sithandra1	Dial	SIP/6217017172813@fwd5
10/11/2020 12:23	drre_9676	17017172813	72.7	0.0144	1.05	ANSWERED	drre_9676	sithandra1	Dial	SIP/6217017172813@fwd5
10/11/2020 8:54	drre_9676	17017172813	280.9	0.0144	4.04	ANSWERED	drre_9676	sithandra1	Dial	SIP/6217017172813@fwd5
10/11/2020 8:54	drre_9676	17017172813	280.4	0.0144	4.04	ANSWERED	drre_9676	sithandra1	Dial	SIP/6017017172813@fwd17
10/11/2020 8:54	drre_9676	17017172813	281	0.0144	4.05	ANSWERED	drre_9676	sithandra1	Dial	SIP/6217017172813@fwd5
10/11/2020 13:15	drre_9676	17017172813	20.2	0.0144	0.29	ANSWERED	drre_9676	snakeeyes1	Dial	SIP/6217017172813@fwd5

avayax · October 12, 2020, 2:28pm

Are you using the FreePBX firewall at all?
It’s not clear from your post what you did to secure your system.
Obviously it’s misconfigured because otherwise these fraudulent hacking attempts would not be getting through our would be banned after a few failed tries.

Drrehak · October 12, 2020, 2:29pm

responsive firewall is on

BlazeStudios · October 12, 2020, 3:11pm

You need to pay attention to this because this is your issue. Your firewall is not configured right. I was also able to hit your GUI and SSH without issue. Your box is exposed to the entire world.

Drrehak · October 12, 2020, 4:20pm

I agree.

Sangoma support walked me through a more secure setup.

Interface should be “Internet”
Firewall services set to Local (i dont use UCP)
Firewall extra services - changed TFTP to Local
Provision protocols. - disabled TFTP server, http authentication to both

changed secret on all extensions
changed web gui password
confirmed intrusion detection running

jfinstrom · October 12, 2020, 6:05pm

There is no obligation to respond at all if you feel it not worth your time.

To quote a wise man… “If you want attention go get a puppy”

Everyone starts somewhere and there is no reason to be toxic.

lgaetz · October 13, 2020, 12:01am

It’s unfortunate that you had to find this out the hard way, but glad to hear you’re sorted. Please take a moment and familiarize yourself with the firewall config and concepts, so that you fully understand where you went wrong.

Wiki: Sangoma Documentation
Video: Open Source Pro Tips #2 - Firewall Basics

fceledon · October 16, 2020, 3:52am

I know how frustrated you feel. I know that feeling very well. Lucky you that is ‘just’ a pbx: ransomware is boiling everywhere.
My few cents to mitigation.

About sip provider. I have a low amount per refill (actually just $30 or any amount that meets your daily use) with a limited number of refills per day (just one): if this number is exceeded I will receive an mail notification, I will know that
a) One of my users was hacked and his extension is being used for calls or
b) The PBX was breached.
c) Business is doing great!! increase the refill amount.
SSH: I have SSH with no password, only SSL key. Not difficult to setup and very effective. To stop the infinite login attempts, limit port 22 to a few familiar know IPs or networks.
HTTP: same as above. Limit who can do it.
SIP/PJSIP ports: Why more than a couple of password attempts? I block the IP after the second failure. No exceptions. Ahh you are blocking a legal user? get your IP from any get your ip web and manually remove the lock. 99.9% of the user leave the password on the app or phone, so only a hacker will ‘try’ different options.
MYSQL: only access from 127.0.0.1 Note: this is a VERY easy way to access your PBX.
Use a virtual machine for your PBX, even an small one. take frequent checkpoints: if you are in trouble, just go back to the last good one. Checkpoint after every change. Before you go back, inspect (or save) the logs for the autopsy.
Use Fail2Ban is very effective and default rules accomplish a lot.

AdHominem · October 16, 2020, 5:57am

I manage ten FreePBX systems. I keep the ports on my router closed to external traffic, and configure IPTables to limit everyone’s local access to needed ports (phones get 5060 and 10-20,000), and haven’t been hacked ever in the nine years I’ve been doing so.