I’ve been playing with the firewall blacklist entries lately on my own PBX and have seen a significant drop of Fail2Ban notifications. I’ve taken to banning /16 networks when I see several attempts and so far no issues.
I suspect the blacklist data is kept in the mysql database somewhere and was wondering if I could somehow write a query in the CLI that could mass update this table with my known bad networks. If I can do that then I can theoretically find a way to automate pushing this out to all my client PBX’s.
# fwconsole firewall add blacklist 1.2.3.4/16 5.6.7.8
Attempting to add '1.2.3.4/16' to Blacklist ... Success!
Attempting to add '5.6.7.8/32' to Blacklist ... Success!
# fwconsole firewall list blacklist
All blacklisted entries.
1.2.0.0/16
5.6.7.8
# fwconsole firewall --help
Usage:
firewall [options] [--] <cmd> [<opt>] [<ids>]...
Arguments:
cmd Command to run (see --help)
opt Optional parameter
ids IDs to add or remove from a zone
Options:
-f, --force Force Add/Removal of entry
-h, --help Display this help message
-q, --quiet Do not output any message
-V, --version Display this application version
--ansi Force ANSI output
--no-ansi Disable ANSI output
-n, --no-interaction Do not ask any interactive question
-v|vv|vvv, --verbose Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug
Help:
Valid Commands:
disable : Disable the System Firewall. This will shut it down cleanly.
stop : Stop the System Firewall
start : Start (and enable, if disabled) the System Firewall
restart : Restart the System Firewall
lerules [enable] or [disable] : Enable or disable Lets Encrypt rules.
trust : Add the hostname or IP specified to the Trusted Zone
untrust : Remove the hostname or IP specified from the Trusted Zone
list [zone] : List all entries in zone 'zone'
add [zone] [id id id..] : Add to 'zone' the IDs provided.
del [zone] [id id id..] : Delete from 'zone' the IDs provided.
fix_custom_rules : Create the files for the custom rules if they don't exist and set the permissions and owners correctly.
sync : Synchronizes all selected zones of the firewall module with the intrusion detection whitelist.
f2bstatus or f2bs : Display ignored and banned IPs. (Only root user).
When adding or deleting from a zone, one or many IDs may be provided.
These may be IP addresses, hostnames, or networks.
For example:
fwconsole firewall add trusted 10.46.80.0/24 hostname.example.com 1.2.3.4
ok, so now I just have to write a script that downloads a list of IP’s, pumps them into that command and then run them on a cron job. piece of cake… It has been so long since I did something like this my brain is gonna hurt…
Luckily the fwconsole command doesn’t add duplicates so every time it runs it just adds any new IP’s I update the pbxblacklist.txt file on my server with ftp client and let it run.
So all pbx’s updated last night at midnight and this morning my inbox only had 65 Fail2Ban warnings. I’ve added those IP’s to the file and pushed it up to the server so the PBX’s will update tonight.
Typically there are hundreds of Fail2Ban notifications a day. so looks like it’s working… we’ll see.
So ran into a couple glitches… the wget wasn’t grabbing a full file, I suspect my web server was caching files or some weird thing… Then realized if I put a bad IP on the blacklist, I’d need some way to remove it automatically… So switched to FTP and a single bash script… This seems to be working well. Simply maintain two files, pbxblacklist.txt and pbxwhitelist.txt. If you need to remove an IP or host from the blacklist, simply modify the file and upload it to your ftp server. then add that IP or host o the pbxwhitelist and upload it as well.
Eventually the script will run, download the blacklist and update the firewall. This won’t remove the IP or host however so the next step is to run a del command with the IP’s/hosts listed it the pbxwhitelist.txt file.
Then set you asterisk crontab to run the bash script once a day on all the servers and it will update your firewall blacklist. Again, I’m sure someone here can do something more elegant than me but it does the job…
#!/bin/bash
Blacklist="pbxblacklist.txt"
Whitelist="pbxwhitelist.txt"
hostname="your ftp server here"
username="your ftp username"
password="your ftp password"
dir="/home/asterisk/"
ftp -in $hostname <<EOF
quote USER $username
quote PASS $password
lcd $dir
binary
get $Blacklist
get $Whitelist
quit
EOF
/usr/bin/cat /home/asterisk/pbxblacklist.txt | /usr/sbin/fwconsole firewall add blacklist `xargs`
/usr/bin/cat /home/asterisk/pbxwhitelist.txt | /usr/sbin/fwconsole firewall del blacklist `xargs`
it’s possible but unlikely. the 45.120.0.0 is a company in Austin Texas but it appears to be a VPN company so who knows where the hackers are really located.
every other day it was a new IP from the 45.120.x.0 subnet… since they were all coming from the same company (or a sub company of that company) and since I tried multiple attempts to get in touch with their support/admins/security people to no avail… It was just easier to block the entire class B.
Since they are also a VPN company I deduced that the attackers were always the same, just using new VPN connections every few days so they kept hopping IP’s.
I know of no legitimate connections that should come from that network for my clients/users so I really wasn’t worried about it.
New fun… Looks like Microsoft has something running around their network. Just had several of my systems all report fail2ban’s on a slew of IP’s that resolve back to Microsoft.
I’ve reported several IPs to Microsoft that were trying to connect to our PBX. MS didn’t care, their response was:
The activity reported is associated with a customer account within the Microsoft Azure service. Microsoft Azure provides a cloud computing platform in which customers can deploy their own software applications. Customers, not Microsoft, control what applications are deployed on their account.
So I blocked huge swathes of their IPs.
My procedure is more manual, when I get a F2B email I look up the IP address here
and then if necessary get the subnet info from here
I kind of suspected that would be the case. Really couldn’t see actual microsoft systems being taken over like that but one never knows… Look at the current Rackspace debacle. (not that they are anywhere near Microsoft…)
One bad apple in a subnet is not indicative , more than , one suspicious.
Tthe smaller guys might have a /24 or less, the cleverer ones populate bigger C&C nets but still generally though one ASN , one for the grunts, one for the general officers.
Do even your bgp ‘prefix’ is often not enough.
(There are no legitimate voip routes coming from Microsoft nor Digital Ocean etc. unless you personally built one. )
Iptables blacklists are slow without ipsets.
N ot accepting UDP/5000-5999 will reduce your drive-bys by ‘a big lot’.
Not accepting IP connections but only strict SNI will make that ‘big lot’ to a ‘huge lot’.
You will need a ‘reverse proxy’ to answer your connections, ones that try to connect to your IP are dropped, those that connect to your FQDN where you have a certification (the certification process is handled by the proxy) that accepts the connect, will be forwarded to your PBX which won’t then need LetsEncrypt or similar anymore , personally I use HAProxy. Depending on your Provider, you might need ‘exceptions’ if they can’t or won’t send calls to your FQDN , (think SRV assignments) (If so, I would look for a better one) extensions are then completely under your control and that’s what you currently struggle with.
