Urgent Help Required (fail2ban failure)


(Paul Hadley) #21

I did consider an absolute IP address but I do have a couple of phones with more that one extension number so that solution would fail in that situation.


#22

Or you could just not use UDP/5060


(Paul Hadley) #23

I have used an off port in the past when I had an Axon PBX (if can remember software that old) that didn’t have any protection. Might make sense thinking about it, can’t really see any obvious drawback?


#24

I know it goes against standard dogma, but I don’t use fail2ban for any clear text SIP traffic.

At iptables my basic SIP flow is:

  • Geoblock most of the world. Quick and efficient dropping of 90%+ of queries
  • A couple of sip blacklists - more efficient than looking into the packet. Gets a meaningful percentage of what gets by geoblocking
  • String matches for a valid “public” extension, valid domain name, and allowed user agent
  • Anything not matching the above is dropped silently - 99.9% of all invalid hits don’t think the udp port is open at all. The few that try tcp (where I have it open) will see the port open, but likely rejected upon the first data packet.
  • Very tight iptables “recent” rules
  • Finally allowed through to asterisk

#25

My journey has taken me through ‘large bills due to fraud’ , adding a solid iptables firewall in my case CSF, good and has native support for geoips and blacklists , BL’s are large, need to add ipsets, not dynamic, add fail2ban, better but a pita to keep up to date, asterisk adds ‘security logging’ . end result was quite good but needs constant log watching to stop ‘big bills’

Re-think, 99.9999% of all this is to my ip using udp on port 5060, revelation, stop doing that sh1t, answer,

Dont use udp, trivial and your phone batteries last longer using TCP, connection still coming but most don’t get to asterisk

Don’t use any port between 5000 and 5999 because after 5060 comes 5080 (freeswitch) 5160 (FPBX) then ‘effit, we will port scan this dude’ (CSF can catch port scans)

domain name only, no IP’s , yes it IS possible to look up a likely host name from an IP, but you can trick 'em , use an additional DNS A record exclusively for voip, then add HAPROXY and enforce SNI so your public facing UCP/WEB frontends won’t leek your domain name to an IP based connection to your machine (or more clumsily don’t open 80 .)

So after 15 odd years, I could watch sngrep on any of a dozen machines and all that 5060 is never replied to and slowly fades away, not your Dutch/Icelandic dude though, ad he and his ilk are very persistant. fail2ban never kicks in (for asterisk) , most of these machines are on a large VM host that has simple firewalls ability, about a dozen rules to allow your chosen port (and 5061, TLS has never been attacked on any of my machines yet) a few whitelist management networks and the occasional pinholes for providers that can only sing udp/5060. 443/80/25 for the obviously needed services, 8089 fir webrtc, 4445 for fop2. . .

One simple firewall, all servers almost totally ‘noise free’ Same simple firewall on your router perhaps?

Now I am almost happy, no dumbass VPN’s needed, clients can go to China or Russia even some middle eastern places that forbid 5060

I wil keep on watching but increasingly feel like the Maytag dude, yep , that’s when they get you, “All’s quiet on the Western front . . .”

So, just try it, it only takes a couple of hours to prove it works and how easy it really is.


(Paul Hadley) #26

I have been fortunate, lots of attacks but I never been hacked.

My strategy to date has been

  1. All outbound routes locked to PBX extensions only.
  2. Extension with secure passwords (bit obvious but?).
  3. Extensions (but for a very limited few) locked to the local LAN.
  4. Fail2ban running with a 4 attempts in 1 hour ban lasting a months (in reality until a reboot).
  5. PBX’s on a dynamic IP address to a weekly router reboot which gave it a new IP at least once a week.

I do miss the security a local LAN lockdown that PJSIP does not seem to support like Chan_Sip did. I do hope PJSIP comes to support this option going forward.

So a hacker had 4 attempts to guess one of maybe 4 extensions that were open to the WAN and get the right password.

Now I have been thinking about it I really don’t know why I kept using 5060, it’s just the port historically used I guess, so over the next week I am going to move my three PBX’s to another port (a few Polycom config files to write) as an additional measure.

As of today 11.20 local time my problematic server is running with fail2ban blocking IP address’s as it should. I think SterlingPkg has probably identified my issue.

The brief pause in fail2ban running was the starting point. If the service started without any attack and restarted it was fine, but if an attack was in progress when the service paused the volume of attempted registrations and entries into the /var/log/asterisk/fail2ban.log exceeded the volume capability of fail2ban which never caught up allowing yet more registration attempts to get through the firewall. It just escalated from there with the PBX slowing grinding to a halt.


#27

currently supported versions of F2B 0.9 and greater maintain bans in non-volatile sqlite3 database so don’t need to ‘reread on restart’ and you can now ban ‘indefinately’ trouble is as your banned list grows, iptables takes longer and longer to load, use ipsets to mitigate this or use s port with less exposure

pyinotify reduces the read latency by ‘lots’.


#28

Unfortunately most distro users aren’t going to be upgrading past the CentOS packages.


#29

Indeed unfortunate for them, also a confusing decision by Sangoma to not do so in their ‘distro’, it has been 4 or 5 years since 0.9 and we now have 10 and 11(nearly) .For the real brave amongst them,

https://centos.pkgs.org/7/getpagespeed-noarch/fail2ban-0.10.4-1.el7.noarch.rpm.html


#30

I just looked at my distro instance. The old version is actually Sangoma’s fault, not a CentOS/RHEL issue:

[root@newdistro2 ~]# yum list installed fail2ban*
fail2ban-fpbx.noarch               0.8.14-76.sng7                @anaconda/1910

But the epel repo is newer:

[root@newdistro2 ~]# yum list available fail2ban
ail2ban.noarch                     0.11.1-9.el7.2                      sng-epel

WHY???


(Paul Hadley) #31

Yes, I tried to an upgrade to fail2ban as part of my repair process and got a report back saying that the fail2ban install was locked by Free PBX?


#32

Stuff like this is why I don’t use any of the FreePBX firewall/IDS even on distro machines.


(Paul Hadley) #33

Just changed my first PBX off 5060, be interesting to see how many fail2ban IP entries I get now?


#34

You can watch the attempts that don’t even get to Asterisk/fail2ban with sngrep, If indeed no 5060/UDP then pretty sure you will see very few get to fail2ban


(Paul Hadley) #35

Well I think I am kicking myself for missing the obvious and not moving my system away from port 5060 for all these years.

Running for over a week now and nothing for fail2ban to do?

Many thanks to all those that replied to my original message and helped me understand why fail2ban was failing at times, much appreciated.


#36

Excellent, and that will almost always be the case.

So, as a new ‘never5060er’ , could you say how hard was that on a scale of 1 to 10?

This so others might be tempted to try, especially as the distro’s fail2ban is increasingly showing it’s frailty.


(Paul Hadley) #37

On a scale of 1 to 10 somewhere between 1 - 2 I think, had a few polycom config files to do but did that using the replace feature in a text editor. Less that an hour off line time to do 3 PBX’s.

I think I went 5060 because it was the default and back in the day when I started on VoiP in maybe 2004 / 2005 there were occasionally programs that didn’t play nice “off port”.

I would recommend anyone using 5060 moves to an off port configuration, in fact I would urge Sangoma to introduce a script into the install and set up routine so users select a signalling port with a recommendation not to use 5060 unless absolutely necessary.


(Michael Jones) #38

lol poor F2B! like retirement!


(Paul Hadley) #39

Wouldn’t remove fail2ban for anything, it has with only a couple of blips kept my system secure for a few years now, but if you can configure an extra (solid) layer of protection in with an off port set up why wouldn’t you?


(Yois) #40

I just compiled a new RPM for FreePBX, upgrading fail2ban to 0.11.1. It works for me, but I’m asking others to test it if they would like.
The SRPM and RPM files are here:
Yois / firewall-fix / ae40a6f43e9 - FreePBX GIT