Unable to ping external IP addresses after NAT IP change

General Help
1

We attempted to change the external IP address on our router and after we did, the UC40 PBXact server could no longer ping any external IP address. Other devices on the network were able to ping external IP addresses without issue.

Literally the only thing that changed was the external static IP for the router. All other devices on the network were able to access the Internet fine.

The UC40 was able to ping the local gateway/router and the router was able to ping the UC40 without issue.

As soon as we changed the address in SIP Settings back to the original IP and changed the router back to its original IP, the server could once again ping external IP addresses.

The reason for the IP change was we are adding a Bigleaf device that allows us to have multiple ISP circuits essentially to provide for redundancy. We are also in contact with them in regards to what may be going on but since the only server affected was this UC40 server I wanted to see if anyone else had similar issues or a potential resolution.

2

Just to do some brainstorming:

UC40 is in DMZ, but the routes are not changed to reflect the new extIP?
So if the UC40 can ping the router and backwards, this indicates a router issue …

So if you say “UC40 can’t ping” - do you mean a normal ping from any kind of console on the UC40 side? Your description is a little irritating, since the IP you use in SIP settings should not have to do anything with the network connection itselfe…

3

It’s not in a DMZ but rather NAT’d with some rules for port 5060 and Twilio’s signaling servers on 10000-20000. Otherwise, the server has full access to the Internet. Thus a ping should be allowed. Again, this the only server on the LAN with this issue so it would be a bit strange that this server needed any special routing on the router.

Yes, I’m using a normal ping from the console on the UC40. I’m logging in via SSH Putty into the console and trying to ping an external IP like the external gateway. I agree, the IP address in the SIP settings should only affect my SIP trunking and not an OS native app like ping. That’s why it had me scratching my head so much.

4

Check the status of the integrated firewall. it’s possible that something has lost it’s mind in there.

5

The integrated firewall is disabled.

6

post the output of:-

ip route
ip address
iptables -L -n
traceroute ip.of.what.ever

from bash.

7

The following results are from the working configuration. I can’t run those commands until off hours and after switching back to the new configuration.

------results from iptables -L -n ------------------

Chain INPUT (policy ACCEPT)
target prot opt source destination
fail2ban-FTP tcp – 0.0.0.0/0 0.0.0.0/0 multiport dports 21
fail2ban-apache-auth tcp – 0.0.0.0/0 0.0.0.0/0 multipor t dports 80
fail2ban-SIP all – 0.0.0.0/0 0.0.0.0/0
fail2ban-SIP all – 0.0.0.0/0 0.0.0.0/0
fail2ban-BadBots tcp – 0.0.0.0/0 0.0.0.0/0 multiport dp orts 80,443
fail2ban-SSH tcp – 0.0.0.0/0 0.0.0.0/0 multiport dports 22
fail2ban-recidive all – 0.0.0.0/0 0.0.0.0/0

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination

Chain fail2ban-BadBots (1 references)
target prot opt source destination
RETURN all – 0.0.0.0/0 0.0.0.0/0

Chain fail2ban-FTP (1 references)
target prot opt source destination
RETURN all – 0.0.0.0/0 0.0.0.0/0

Chain fail2ban-SIP (2 references)
target prot opt source destination
RETURN all – 0.0.0.0/0 0.0.0.0/0
RETURN all – 0.0.0.0/0 0.0.0.0/0

Chain fail2ban-SSH (1 references)
target prot opt source destination
RETURN all – 0.0.0.0/0 0.0.0.0/0

Chain fail2ban-apache-auth (1 references)
target prot opt source destination
RETURN all – 0.0.0.0/0 0.0.0.0/0

Chain fail2ban-recidive (1 references)
target prot opt source destination
REJECT all – 62.210.74.151 0.0.0.0/0 reject-with icmp-po rt-unreachable
REJECT all – 207.38.90.197 0.0.0.0/0 reject-with icmp-po rt-unreachable
REJECT all – 151.106.13.126 0.0.0.0/0 reject-with icmp-po rt-unreachable
REJECT all – 151.106.13.171 0.0.0.0/0 reject-with icmp-po rt-unreachable
RETURN all – 0.0.0.0/0 0.0.0.0/0

ip route

10.0.2.0/24 dev eth0 proto kernel scope link src 10.0.2.26
169.254.0.0/16 dev eth0 scope link metric 1002
default via 10.0.2.1 dev eth0

ip address

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:90:27:ee:fd:b2 brd ff:ff:ff:ff:ff:ff
inet 10.0.2.26/24 brd 10.0.2.255 scope global eth0
inet6 fe80::290:27ff:feee:fdb2/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 00:90:27:ee:fd:b3 brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 00:90:27:ee:fd:b4 brd ff:ff:ff:ff:ff:ff
5: eth3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 00:90:27:ee:fd:b5 brd ff:ff:ff:ff:ff:ff

When I ran traceroute it showed * for each hop. It never showed the default gateway for example.

8

When you can do that, please blockquote them , also edit your post to blockquote the bits you see are badly formatted.

you can blockquote by highlighting the relevant bits and choosing that icon, pressing ctrl+shift+9 or adding triple back quotes before and after the block

I would surmise after looking again, that the device you have at 10.0.2.1 is your underlying problem, it apparently drops ICMP, so what exactly is it and is it trying to “help you” with SIP connections (ALG)

9

10.0.2.1 is the gateway. From any other device on the network, I can ping any external IP I want. When I put the gateway back to the original WAN IP address, the UC40 can once again ping any external IP. I have no rule that specifically allows or denies ping traffic.

10

I realize its the gateway, the question was “what is that device ?”

ICMP (ping and traceroute use it) is a layer 4 (ethernet) protocol so not IP based, ARP translates MAC address (Layer 3) to IP address (layer 4) , so it is possible that you need to flush the arp cache on your firewall/router a SIP “Helper” or other feature in the router might remember the state of things from before you changed those things.

11

It’s a SonicWall TZ 300. On the firewall, the LAN to WAN rules has exactly one rule, Allow-Any Source to Any Destination for Any Service.

12

Ping needs ARP to properly resolve MAC address as I said, did you flush the current entries on the SonicWall? Maybe just turn it off for a minute or two, did you disable the SIP ARG?

13

Yes, I tried to power cycle the SonicWall AND the cable modem and DSL modem. Usually when it’s an ARP table issue, nothing on the LAN can get out since the MAC of the router isn’t in the ARP table of the modem(s).

Under VOIP settings, Enable consistent NAT is not checked. Enable SIP Transformations is NOT checked. Enable H.323 Transformations is also not checked.

14

Did you check your LAN to WAN rules?

Btw, Sonicwall has a great network capture tool, you can filter for any traffic going from the PBX to the www. That should give you clues.

15

OK - time for pedantic mode.

Even with the SonicWALL in place, this shouldn’t be this hard.

Also, you keep saying “ping”.

Do you really mean the system command ‘ping’ or are you mistakenly using that to say that you’re connectivity is a problem? It’s important because they are completely different problems, even though without the ability to ping, nothing else is going to work. When you say ‘ping’ to me, it means something very specific. Many people use it as a more vanilla term, but here, a lot of us are old work-horses and we need words to mean what they mean.

The ping command has everything to do with IP connectivity and routing and nothing to do with SIP, so until you get the ping command part working, trying to get SIP working is definitely a non-starter. This is an ‘eating an elephant’ problem - you need to know what’s working and solve each non-working problem one-at-a-time.

Without knowing what your settings in the “new” configuration are, there’s no point trying to troubleshoot this. Since you know the working configuration is working, the new configuration should be very close to the same.

When you get the new configuration loaded again, you need to make sure you have network connectivity throughout the network. You need to start by pinging the local 10.0.2.x network servers that are important to your setup. If you can’t get to those, check your netmask. Start with your own PBX server - ping the local LAN address of your “localhost” machine. This makes sure that nothing crazy is happening in the local interface configuration. You can also use ‘arp -na’ from the command line to make sure the ARP addresses on all of the servers are visible.

Once you are certain that the server can see and be seen by all of the other machines in the local network, trying pinging outside the network. Many devices are set up to not respond to ICMP packets, especially backbone equipment - these will show up as ‘*’ in the transcript.

On the PBX, you can (from the console) use ‘tcpdump -I eth0’ to watch the traffic as you do your tests. Make sure that everything that goes out makes sense. From this, you should be able to troubleshoot the local network issues, if you are having any.

There’s a really good how-to on using the SonicWALL router on the forums. Have you looked at that?

16

Yes I mean ping as in “ping 8.8.8.8” or “ping www.yahoo.com” or “ping [external gateway IP]” from the command line of the UC40 when using Putty SSH.

So as far as connectivity goes, I’m able to ping, using the centos ping command, the internal gateway, any LAN IP, and I can ping, using the GUI, on the Sonicwall to the UC40. So connectivity between the UC40 and the Sonicwall gateway seems to be fine. Simply changing the WAN IP on the SonicWall gateway seems to break the ability for the UC40 to be able to communicate with any external IP address. If I move the WAN IP back to the original IP, it all works fine again.

So the settings in the new configuration on the SonicWall gateway is a new WAN IP address, a new WAN subnet mask, and a new WAN Default Gateway. That’s all that is changing and even if I do nothing else on the UC40 like change the external IP in the SIP settings, which should have zero affect on the centos ping command, I’m unable to ping any external IP address. I can ping from any other device on the LAN to any external IP address. Users were able to browse external web sites without issue while on the new WAN IP.

The way I first noticed this was that it wasn’t able to bring the main Twilio trunk up because it identifies it by host name and it wasn’t able to resolve the host. I thought at first it was a DNS issue but when I used the centos ping command to ping 8.8.8.8, it timed out. That’s what led me to believe that all traffic to the WAN is being blocked somehow.

When I change the WAN IP info on the SonicWall, I’m still able to use the centos ping command to ping 10.0.2.1 or any other valid IP on the LAN. I’m also able to use the centos ping command to ping the IP of the UC40 without issue. IOW, the UC40 can ping itself (e.g., ping 10.0.2.26).

Yes, I have looked at the Sonicwall guide. Remember, this is a configuration that has been working for two years on the Sonicwall without any change in configuration up until we changed the WAN IP settings on the SonicWall.

I’ll try the tcpdump command when we give this another go later this week.

Thanks for the reply.

17

One other tidbit, we have an IPSec VPN between the SonicWall and another SonicWall at our corporate office. When I changed the WAN IP on the Sonicwall and then change the IP address for the VPN on the corporate office VPN configuration, the VPN comes right back up and I’m able to ping any address on that remote LAN. I haven’t tried pinging a device on that remote LAN from the UC40. I’ll try that next time as well.

18

Yes, I did check my LAN to WAN rules and there is only one rule that allows any source to any destination for any service. Again, all I have to do is revert the WAN IP settings on the SonicWall back to the original WAN IP settings and everything is back to working.

What network capture tool are you referring to? Some of that I’ve found are add ons that must be purchased.

Thanks.

19

This has got to be a limitation of or a problem on the SonicWALL. There’s nothing in an ICMP packet that will do that. As far as the UC40 and ‘ping’ are concerned, the internal LAN address on the SonicWALL hasn’t changed and the outbound traffic from the UC40 has to gateway through that connection.

To double check that, try a ‘netstat -nr’ and see what your routing rules are. Unless they are referring to the external address of your SonicWALL, there’s nothing in this layer of the IP network that has any idea how the traffic is getting out and back in.

Finally, double check your NAT settings in the SonicWALL.

Like I said before, this shouldn’t be this hard.

20

I agree, it shouldn’t be this hard. But here I am.

Here is what I get from netstat -nr

Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
10.0.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
0.0.0.0 10.0.2.1 0.0.0.0 UG 0 0 0 eth0