Firewall Issue - Old Phones & TFTP

[RealRuler2112 (Ruler2112)]

This is going to be a very long shot since Lorne wasn’t able to resolve it & he seemed very, very knowledgeable.

In short, I have two PBXact 13 appliances configured in HA mode. Due to our users familiarity and comfort level with our existing Cisco phones and overall cost involved with replacing 70+ phones, it was decided to keep the old Cisco 7960 phones, flash them to SIP, and utilize them with our new phone system for the majority of our phones; basically, managers & our call center got Sangoma phones - everyone else has a 7960. After I figured out how to do this, setting them up went relatively smoothly. (Figuring out how to flash them & the exact config file structure/content the phones needed was the hard part. On a side-note, would anyone be interested in a how-to style guide on how to do this?) Being old, they provision using TFTP and I have option 66 enabled on our DHCP server to point them to the cluster IP address. This went well and the phones work as expected with PBXact.

However, while configuring the server, I noticed something very odd.

Whenever I factory reset a 7960 to remove all settings from it, it will not pull it’s config from PBXact. I can see the TFTP connection coming into the server using a packet sniffer, but the server refuses to respond and there’s nothing in /var/log/messages from the tftp daemon, even though verbose logging is turned on. If I disable the firewall temporarily from the command line (service iptables stop), the phone receives it’s configuration and registers perfectly. The firewall restarts after a 5-7 second delay, but this is more than enough time for the phone to pull it’s config. After a given phone has registered once, it can then boot up normally without needing to disable the firewall to let it through. If a functional phone is factory reset though, it is again not allowed through the firewall and refuses to pull it’s configuration until I manually disable the firewall.

If I change the eth0 to ‘Local (trusted traffic)’ instead of ‘Internet (default firewall)’ under Firewall->Main->Interfaces, this problem does not present itself. Obviously, this is not something I wish to run with long-term. I have already set the phone subnet /24 to be ‘Local (trusted traffic)’ under Firewall->Main->Networks, but this does not seem to have any effect - the ENTIRE interface needs to be set to ‘Local (trusted traffic)’ in order for the phones to provision.

I tried disabling the responsive firewall in Firewall->Main->Responsive Firewall, but this had no effect either - the entire firewall needs to be disabled or have eth0 be set to Local in order for them to provision. I also set TFTP to be allowed in the Internet zone in Firewall->Services->Extra Services, but alas this did not change the symptoms either. There are 0 ‘Rate Limited’ or ‘Blocked Attackers’ shown in Firewall->Status->Blocked Hosts. I also added the entire subnet to the whitelisted clients under System Admin->Intrusion Detection on the off-chance that this is what was causing it.

The last thing I tried was to upgrade the firewall module from 13.0.57.1 that was on the system when installed to edge release 13.0.60.2, then re-save & apply, but again this had no impact.

After opening a support ticket, Lorne found out that apparently, I’m affected by a bug that’s supposed to be fixed already:

https://issues.freepbx.org/browse/FREEPBX-14483

Unfortunately, nobody seems to know how it was fixed in the past. I tried loading the TFTP connection tracking module as the bug report suggests, but this had 0 impact on the issue.

Thankfully, my PBXact system is protected by a Meraki edge device, so I can use that to shield it from the internet & leave the entire interface configured as being completely trusted/local. (It was this or disable the firewall entirely.) I would much rather have the firewall on & functioning as it should to provide another layer of protection, but simply do not have the knowledge/experience to do so; the fact that I set TFTP to be allowed in the Internet zone and it still won’t work makes me think there’s a flaw in the GUI or firewall rule generation. Lorne went back & forth with R&D and a decision was made to not spend any more time on it.

I was wondering if anyone here has experienced this, has a solution, or is a wizard at iptables and could perhaps provide an auxiliary rule that might alleviate this issue. (Frankly, it’s been years since I used iptables; I use BSD for most of our servers & so know/like pf MUCH more; it’s so much simpler and easier IMO. Looking at the iptables output on the PBX makes me slightly dizzy… )

[mulderlr]

I don’t recall a 7960 using dhcp option 66, and have only ever gotten them to work with option 150. So, I am not sure how you are even seeing tftp traffic in your pbx from a factory reset 7960, but I would suggest setting up dhcp option 150 for them and see if that helps.

[cynjut (Dave Burgess)]

@mulderlr

I don’t recall a 7960 using dhcp option 66, and have only ever gotten them to work with option 150.

I just checked my dhcpd.conf file (I use ISC DHCP on a NetBSD server) and I set both Option 66 and Option 150 to point to the TFTP server IP address. The phones don’t want a URL - they just want the IP address. I use 7940 and 7960 phones almost exclusively, and this works for me. Opening both 66 and 150 as Local “additional” services in the Firewall Module worked fine for me. I also open port 2000 and use Chan-SCCP-B to get around the glitches and differences between the SIP image and the original SCCP image.

[mulderlr]

dhcp options are not firewall ports. They simply pass parameters to the client from the DHCP server to help it locate a specified service. And you are correct the cisco’s don’t want a URL, they just need an IP address because they will only provision via TFTP and nothing else, meaning no other provisioning protocols are supported on the SIP firmware for those devices. So, if you have option 150 defined (it is a custom definition in almost every DHCP server), and then specify the IP address of your PBX or where your cisco 7960 config files are located, you shouldn’t have any trouble with them picking up the TFTP IP address and hence the configs assume you have things in the correct place.

This being said, If the phones are on the same LAN segment as the PBX, they shouldn’t have to go through a firewall to get to the TFTP server they need to connect to except and unless you are using the software firewall (if any) on your PBX. In that the only ports the you need to worry about for tftp provisioning is 69/udp. If that port is open, and option 150 is set up correctly, you should see the phones (assuming they are running sip firmware) pull their configs or at least request them via TFTP.

[RealRuler2112 (Ruler2112)]

I flashed the 7960 phones to SIP, so they’re not running SCCP anymore. (Tried for about 2 weeks to get them to work with SCCP firmware with PBXact, but never did figure out how to do so. ) After they were running SIP, they provisioned with DHCP option 66 fine. The DHCP pool for the phone subnet is set to serve just the IP of the phone system using option 66.

The problem isn’t with the phones getting the IP of the boot device though, it’s that the firewall on the phone system blocks the TFTP connection from freshly-reset phones unless the firewall config has the interface set to be Local. With the interface in the Internet zone, if I take a functioning 7960 and do a factory reset on it by holding # when it boots & then entering 123456789*0# followed by 2 (to reset the network configuration of the phone as well as the rest of the config), it will never register. It gets the TFTP server it should use via option 66 and I can see the requests coming into the server from the phone using tcpdump, but the requests never make it through iptables. If I stop iptables, the phone registers almost instantly.

The entire /24 subnet of the phones is whitelisted (I tried both local & trusted/excluded in the firewall module) and I even opened the TFTP protocol in the Internet zone. Both the phones and the servers are on the same network segment/subnet/ip range/VLAN and it happens even with both a phone & the server on a dumb/unmanaged switch set up in a test configuration on the bench in my office.

It’s truly mind-boggling as to why the firewall is blocking TFTP requests from a whitelisted IP range…

[mulderlr]

so basically you have already figured out that the issue with the phones provisioning is a firewall issue on pbxact? I don’t deal with pbxact, but this is why I do my own firewall rules - I am not leaving the security of my appliance in the hands of a buggy application, but realize this is not reality for most people. You could try to just running an iptables command to allow DHCP, but depending on the HA config, you would want to take that into account. I would be inclined to say disable the firewall or shut it off temporarily until all your phones are provisioned and then if you want to run in production with it on, you only have to worry about it when/if you re-provision older cisco phones.