FreePBX Intrusion Detection

General Help
1

I am running distro version 3.211.63-7 with Asterisk 1.8.21.0
My system is set to NOT allow Anonymous Inbound SIP Calls
but to allow SIP guests.

Looking at my logs I see thousands of attempts to break into the system, trying one sip extension variation after another.
Fail2ban is not blocking the IPs (I guess) because the Asterisk log does not show the incoming IP when receiving incoming SIP connections from an unknown peer

However a simple “SIP set debug on” command or “sip show channels”, shows the originating IP address.

Given that SIP security is important and a breach could be very expensive and given that the attackers IP is available to the system why can’t we block it automatically? I am not technical enough to do this but it appears to me that it should be possible to get the info and use it in fail2ban or some other application. I manually blocked the IP range and the attack stopped. Here are my findings.

<— Transmitting (NAT) to 83.170.84.47:5074 —>
SIP/2.0 200 OK
Via: SIP/2.0/UDP 83.170.84.47:5074;branch=z9hG4bK-a1c25859cdd39128e084bf31113392f5;received=83.170.84.47;rport=5074
From: 3182sip:[email protected];tag=13aaaca0
To: 001441904891988sip:[email protected];tag=as0bd0f7d6

SIP SHOW CHANNELS
83.170.84.47 3197 122cee76f85f76c 0x0 (nothing) No Rx: BYE
83.170.84.47 3194 9a3d8e3234fe1d3 0x0 (nothing) No Rx: BYE

2013-04-14 19:44:35 1365983075.626 3134 Answer s (from-sip-external) ANSWERED 00:00
2013-04-14 19:44:25 1365983065.625 3133 Answer s (from-sip-external) ANSWERED 00:00
2013-04-14 19:44:14 1365983054.624 3132 Answer s (from-sip-external) ANSWERED 00:01
2013-04-14 19:44:04 1365983044.623 3131 Answer s (from-sip-external) ANSWERED 00:01
2013-04-14 19:43:53 1365983033.622 3130 Answer s (from-sip-external) ANSWERED 00:01
2013-04-14 19:43:43 1365983023.621 3129 Answer s (from-sip-external) ANSWERED 00:00

…
[2013-04-14 19:36:33] VERBOSE[7332] pbx.c: – Executing [001441904891988@from-sip-external:1] NoOp(“SIP/xxx.xxx.xxx.xxx-00000243”, “Received incoming SIP connection from unknown peer to 001441904891988”) in new stack
[2013-04-14 19:36:33] VERBOSE[7332] pbx.c: – Executing [001441904891988@from-sip-external:1] NoOp(“SIP/xxx.xxx.xxx.xxx-00000243”, “Received incoming SIP connection from unknown peer to 001441904891988”) in new stack
[2013-04-14 19:36:33] VERBOSE[7332] pbx.c: – Executing [001441904891988@from-sip-external:2] Set(“SIP/xxx.xxx.xxx.xxx-00000243”, “DID=001441904891988”) in new stack
[2013-04-14 19:36:33] VERBOSE[7332] pbx.c: – Executing [001441904891988@from-sip-external:2] Set(“SIP/xxx.xxx.xxx.xxx-00000243”, “DID=001441904891988”) in new stack
[2013-04-14 19:36:33] VERBOSE[7332] pbx.c: – Executing [001441904891988@from-sip-external:3] Goto(“SIP/xxx.xxx.xxx.xxx-00000243”, “s,1”) in new stack
[2013-04-14 19:36:33] VERBOSE[7332] pbx.c: – Executing [001441904891988@from-sip-external:3] Goto(“SIP/xxx.xxx.xxx.xxx-00000243”, “s,1”) in new stack
[2013-04-14 19:36:33] VERBOSE[7332] pbx.c: – Goto (from-sip-external,s,1)
[2013-04-14 19:36:33] VERBOSE[7332] pbx.c: – Goto (from-sip-external,s,1)
[2013-04-14 19:36:33] VERBOSE[7332] pbx.c: – Executing [s@from-sip-external:1] GotoIf(“SIP/xxx.xxx.xxx.xxx-00000243”, “0?checklang:noanonymous”) in new stack
[2013-04-14 19:36:33] VERBOSE[7332] pbx.c: – Executing [s@from-sip-external:1] GotoIf(“SIP/xxx.xxx.xxx.xxx-00000243”, “0?checklang:noanonymous”) in new stack
[2013-04-14 19:36:33] VERBOSE[7332] pbx.c: – Goto (from-sip-external,s,5)
[2013-04-14 19:36:33] VERBOSE[7332] pbx.c: – Goto (from-sip-external,s,5)
[2013-04-14 19:36:33] VERBOSE[7332] pbx.c: – Executing [s@from-sip-external:5] Set(“SIP/xxx.xxx.xxx.xxx-00000243”, “TIMEOUT(absolute)=15”) in new stack
[2013-04-14 19:36:33] VERBOSE[7332] pbx.c: – Executing [s@from-sip-external:5] Set(“SIP/xxx.xxx.xxx.xxx-00000243”, “TIMEOUT(absolute)=15”) in new stack
[2013-04-14 19:36:33] VERBOSE[7332] func_timeout.c: Channel will hangup at 2013-04-14 19:36:48.814 EDT
…

2

OK, in the absence of any other offered forum I will start here with a stock install of the latest iso . 32 bit and Asterisk 11 but that should be immaterial right now.

let’s get a reasonably secure base environment

export ADMIN_EMAIL="[email protected]"
export SSHPORT=1345 # between 1025 and 65000 perhaps

chkconfig iSymphonyServer off
service iSymphonyServer stop

It’s a bit of a hog, if you want to use it then start it when you set it up.

sed -i “s/#Port 22/Port ${SSHPORT}/” /etc/ssh/sshd.conf
service sshd restart
yum -y install lsof
rpm -Uhv ftp://ftp.pbone.net/mirror/download.fedora.redhat.com/pub/fedora/epel/6/x86_64/rkhunter-1.4.0-1.el6.noarch.rpm
sed -i “s/MAIL-ON-WARNING=”"/MAIL-ON-WARNING="${ADMIN_EMAIL}"/" /etc/rkhunter.conf
sed -i “s|#XINETD_ALLOWED_SVC=/etc/xinetd.d/echo|XINETD_ALLOWED_SVC=/etc/xinetd.d/tftp|” /etc/rkhunter.conf
rkhunter --update --propupd --sk --check

Green is both good and informative, red and yellow things are causes for concern to a sentient admin!

Small changes but I feel we have a stable starting point and any critical changes will be emailed to ADMIN_EMAIL when cron.daily runs rkhunter --update -c if your postfix is working.

There are options to add directories to be included in the rkhunter checks like /etc/asterisk and for a stable system the notifications won’t be too noisy. We can do that later maybe.

More to follow . . .

3

Fail2ban works by parsing logs and looks from specific regex of information. It can only parse logs as far as I know but I am not a expert in fail2ban. So it would need to be in the asterisk log with the IP and match one of the filters.

4

Yes but fail2ban is not the holy grail. I would gladly contribute towards a new module that enhances security beyond what fail2ban offers or that integrates with fail2ban by counting SIP “bye” messages from the same IP and then posts a message in the log that allows fail2ban to block the IP.

I am not a programmer but many in the community are and given that the attackers IP information is available to the system we should be able to block it.

5

Fail2ban parses any log you point it to, the events of concern will be of the type NOTICE or SECURITY. so don’t overwork fail2ban by trailing /var/log/asterisk/full, I suggest you set a log file to only cover those events somewhere in /etc/asterisk/logger*

fail2ban => security,notice

this saves unnecessary work parsing irrelevant log entries.

res_security_log is a relatively recent addition to asterisk and Digium’s docs as are at:-

https://wiki.asterisk.org/wiki/display/AST/Asterisk+Security+Framework

There is a nice workup on working configuration at:-

http://sourceforge.net/p/raspbx/discussion/tutorials/thread/6288a838/

but note that allowguest=no is necessary to use his regex’es effectively.

But to take the subject further, a decent firewall on your box is never a bad thing. csf/lfd from ConfigServer.com has a whole slew of settings to limit and ban IP’s by port and rate of connections, it also serves well as a monitor of any internal runaway applications. Fail2ban doesn’t always play well with other iptable based filters (firewalls) but csf has a couple of scripts, csfpre.sh and csfpost.sh that will allow you to “chain” the fail2ban tables elegantly to it’s own dynamic tables.

Maybe it’s not necessary to reinvent the wheel here, I believe that you could kick /etc/csf/csf.conf (the well self documented config file) into shape for your allowguest=yes scenario.

I too would be happy to collaborate in any opensource security module(s) as I come from @home and am more than 'once bitten twice shy", there is nothing worse than a false sense of security when it comes to your bank account. I would add an effective rootkit monitor for the same reason .

6

Thanks for your comments.
I don’t really need allowguest=yes. The documentation says to set it to yes if you want to see the intrusion attempts. I set it to “no” now and will look at the res_security_log module to enhance my fail2ban regex. My firewall is ok. No one got in and I can’t close port 5060. Maybe the above enhancements will eventually trickle down to the FreePBX distro.

7

I 100% agree that we should look at better tools for BFD. I have been playing alot with CSF lately and we use it on SIPStation servers and it works well.

I think Dicko is a big fan of CSF. Want to help build a CSF module for FreePBX??

8

Of course Tony, if it is opensource ;-).

Pretty easy I would imagine, if, unlike myself one was fluent in php, but it would need elevated permissions to write into /etc/csf so that might be an ongoing problem especially due to the underlying authority of it’s very raison d’etre.

With a carefully crafted suid script and sudo or perhaps at, I believe that can be accomplished without compromising the rest of the system.

9

I am in 100% agreement that it would be open source hence why asking for your help. I think sudo would be to dangerous.

10

As you say, sudo can be “to (sic) dangerous”, as can any suid script, but you guys apparently do something similar with your system admin module, no?.

Thinking outside my old box, one less overt way is to call at with the right at.allow permissions, something like

at -f thescript now + 1 minute

I have never been an advocate of the “ready, shoot, aim” paradigm, this gives me a whole minute to work out whether the call was legitimate or not and kill it if it doesn’t meet the niceguy test, yet still be reasonably close to real time.

If anyone wants to play I’m up for it . . .

11

Well we get a little fancy with how we do things in system admin. The module does not write out anything direct. It puts it all in the database and then using some inotify application called incron it takes the new entries from the database and puts them into the configs so the apache user is not touching the configs direct.

So allows for extraction layer plus you can control what types of information can go into the DB to keep SQL injections from being put in.

12

I appreciate your disclosure Tony, it is basically the same concept, an asynchronous update of the system by a trusted proxy with elevated permissions. The trust is always the crux, can we always trust that procedure or do we just go with blind faith? Further, I will argue, if one propagates what is intended to be system only variables into mysql databases who is to say that mysql can never be penetrated, we actually know otherwise by past experience. That to me is “Danger Will Robinson!” Just one more unnecessary leak waiting to be exploited, make your “dip” into the system ephemeral and make sure you wiped your ass well afterwards.

I trust nothing until I fully understand it, hence my reticence to allow obfuscated code in my systems, it’s just the way I am.

13

Yes, after looking at it and playing with it, incron/inotify fits within my rigorous open-source thinking and would be much closer to realtime:-

Q: Is there a license restriction (due to the GPL-ed kernel)?
No. System calls are considered as “normal use”. It’s similar to use any other system calls such as read() or gettimeofday(). The inotify implementation itself (in the kernel) is of course licensed under GPL and if you want to use this code you must keep the same license.

I assume this module will “keep the same license”

14

We already said this “NEW” module would be GPL. Just need to find Dev time or someone to help donate time to work on this.

I guess I could slip in the GUI work if you can work up all the config files and what options we need to expose to users in the GUI.

15

This is good stuff, it could be a “change SSH port” script from the CLI.

Dicko, open a ticket for a feature request please.

16

So next step:-

Install csf/lfd

cd /usr/src
wget http://configserver.com/free/csf.tgz
tar -zxsvf csf.tgz
cd csf
./install.sh

This gives a basic setup, the interface is directly within /etc/csf/csf.conf, /etc/csf/csfwebmin.tgz is included if you care to install it and like GUIs.

This little script is a first attempt to isolate all the variables into a place where a GUI could be built from.

#!/bin/bash
TESTING=1 # if 1  You get to edit the variables and a trace is left of your work
# extract all the variables into file variables
grep -E "^[A-Z]|^###|^# SECTION" /etc/csf/csf.conf |sed 's/ = /=/' > variables
if [ ${TESTING} -eq 1 ];then nano variables;fi
echo "#!/bin/bash" > csf.local
echo "source variables" >> csf.local
echo "cat <<EOF" >> csf.local
cat csf.conf |sed -e  "s/\(^[A-Z].*\)\( = \)\(.*\)\(\"\)/## \[$(date +%Y%m%d_%H%M)\] \1\2\3\"\n\1\2\"$\{\1\}\"/" >> csf.local
echo "EOF" >> csf.local
chmod +x csf.local
./csf.local > csf.conf
if [ ${TESTING} -gt 0 ]; then
        ./csf.local >  /etc/csf/csf.conf
else
        ./csf.local|grep -vE "^## \[" > /etc/csf/csf.conf
fi

The file “variables” contains 25 sections of which some are useful and some should be default for an asterisk box.

17

Hmm, can’t edit the previous post. .

To get fail2ban to play nice

/etc/csf/csfpost.sh contains

#!/bin/bash
service fail2ban start

and

/etc/csf/csfpre.sh contains

#!/bin/bash
service fail2ban stop

chmod +x both of them.

chkconfig fail2ban off
service fail2ban stop
csf --restart
and you should be up and running.

18

There are as I said 25 sections and 304 variables, there will be many recipes as to how deployed. Dynamic DNS needs one flavor, NAT’ted boxes another, Some will like a permissive Firewall some restrictive. Maybe some can can come up with “templates” that suit themselves and offer these as starting points, some are easier like email notifications and directory watching.

For example if UDP/5060 is generally not allowed, then a quick

rasterisk -x ‘sip show peers’|awk ‘{print $2, “# added by FBX”}’|grep -E “.” >> /etc/csf/csf.allow

would still allow SIP traffic from between your endpoints.

19

Please bear with me as I no longer can edit my posts so Errors and Omissions will have to be accepted :wink:

20

Watch out for that code the [code] tags don’t work, I’ll try again
without:-

#!/bin/bash
TESTING=1 # if 1 You get to edit the variables and a trace is left of your work

extract all the variables into file variables

grep -E “^[A-Z]|^###|^# SECTION” /etc/csf/csf.conf |sed ‘s/ = /=/’ > variables
if [ ${TESTING} -eq 1 ];then nano variables;fi
echo “#!/bin/bash” > csf.local
echo “source variables” >> csf.local
echo “cat <<EOF” >> csf.local
cat csf.conf |sed -e “s/(^[A-Z].)( = )(.)(”)/## [$(date +%Y%m%d_%H%M)] \1\2\3"\n\1\2"${\1}"/" >> csf.local
echo “EOF” >> csf.local
chmod +x csf.local
./csf.local > csf.conf
if [ ${TESTING} -gt 0 ]; then
./csf.local > /etc/csf/csf.conf
else
./csf.local|grep -vE “^## [” > /etc/csf/csf.conf
fi