Asterisk 13.38.2 built by mockbuild @ jenkins7 on a x86_64 running Linux on 2021-04-07 06:49:36 UTC on a Sangoma40
Incoming and outgoing calls just with drop no rings.
I have a dedicated fiber line for SIP with a SIP trunk passing through and SBC.
We had a power outage and now I can’t get the asterisk to register with the SIP Trunk.
I can’t even ping the SBC now. (PBX =192.168.1.3) (SBC = 192.168.1.2)
All my subnets have been whitelisted in the firewall and IPS.
I have contacted the ISP and their engineers have so far not found anything wrong with the SBC
They say they can see my PBX but I can’t see the SBC…
I don’t see anything in the full logs that stands out, but I have only moderate asterisk experience…
Any ideas, I’m trying to rule out everything on my end before I call them back and tell them their SBC is fried?
Setups with an ISP-supplied SBC are generally statically configured and do not use registration or authentication, so lack of registration is probably normal.
Confirm that you are pinging from a shell prompt on the PBX.
Confirm that the SBC is supposed to respond to ping (possibly, its internal firewall is blocking ICMP).
If an attempted ping results in an error message (not merely no reply), post that.
Confirm that ifconfig shows an interface with IPv4 address 192.168.1.3 and an appropriate netmask.
Capture traffic with tcpdump and see whether your attempted pings are being sent out from the interface and whether there is any reply. (The tcpdump capture is after iptables for sent packets, and before iptables for received packets.)
If tcpdump shows ICMP echo (ping) being sent and nothing received, check for a switch or other networking hardware between PBX and SBC. If so, try running a cable directly. Note that depending on the interfaces, you may need to use a crossover cable.
If still no luck, get a laptop and configure its Ethernet interface for static IP 192.168.1.3 and netmask 255.255.255.0. Connect it directly to the SBC with a known good cable. If you still can’t ping the SBC, it is probably indeed fried. However, this conflicts with their claim that they can ‘see’ your PBX.
You are correct. It is statically configured, with the only info required being type=peer and host=SBC IP But it used to show the trunk info under “sip show registry” Now not so much…
[root@freepbx ~]# asterisk -rx “sip show registry”
Host dnsmgr Username Refresh State Reg.Time
0 SIP registrations.
I don’t think I have ever had to run a ping simultaneously with a tcpdump. How exactly is that done run a ping then pipe it into a tcpdump? Run multiple session? Anyway I have a SharkTap ethernet sniffer. I’ll run Wireshark on it tomorrow and will post back results.
Will be on site tomorrow to test this.
The link light on the SBC shows link. but that doesn’t necessarily mean its passing traffic. Also I am off site right now and I just set up a softphone so I could see what they were getting when dialing out. I get no rings only a message that says “all circuits are busy now please try your call again later” then hangs up.
The Host Unreachable is an ICMP error sent from the SBC, presumably because it blocked the ping request. You could confirm this with tcpdump. But in any case, it shows that the connectivity is ok.
If you got the message within a second or two, the SBC rejected the call. There may be a clue in the Asterisk log showing a ‘hangup cause’. However, you can get all the details:
At the Asterisk command prompt (not a shell prompt), type sip set debug on
You should see SIP Debugging Enabled
Make a test call and the Asterisk log will now include a SIP trace. If you can’t figure out what is wrong, paste the complete log for the call at pastebin.freepbx.org and post the link here.
OTOH, if you hear ‘all circuits are busy’ after a long delay, then something (most likely FreePBX firewall) is blocking the request or the reply, and Asterisk gave up after many retries. You can run sngrep to see the SIP traffic on the network side of the firewall.
Sending host unreachable in that context is a bit naughty, although I suppose it could be done to confuse a hacker. Normally it would be port unreachable, or administratively prohibited, for a firewall rejection on a destination machine.
An alternative mechanism for host unreachable, given the destination is on the same sub-net, would be that the SBC failed to respond to an address resolution protocol (ARP) query, which would indicate it had either forgotten its IP address, or the hardware was broken on that interface. The LInux CLI “arp -a” command can be used to see if it recently responded to an ARP request.
I’m not sure if the following will actually work, but you could try pinging 255.255.255.255, to see if anything responds from an unexpected address.
In that case, the SBC either doesn’t think its address is 192.168.1.2, or it isn’t physically connected to the network that it shares with Asterisk. E.g. this is what I get when pinging a non-existent address on my LAN, and then immediately running ARP. Again it says incomplete.
ping 192.168.1.66
PING 192.168.1.66 (192.168.1.66) 56(84) bytes of data.
^C
--- 192.168.1.66 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 41ms
root@dhcppc4:~# arp -a
? (192.168.1.66) at <incomplete> on enp3s0
_gateway (192.168.1.1) at XX:XX:XX:XX:XX:XX [ether] on enp3s0
Sorry, I missed that one. @david55 is correct, and the log posted shows that the call was rejected because no trunks were available so an INVITE to the SBC was not even attempted. I assume that you have qualify set for the trunk and the OPTIONS requests had failed so the trunk was marked offline.
Something you can try quickly is capturing traffic with sngrep or tcpdump and seeing whether anything is sent by the SBC, either periodically or on an attempted incoming call.
From your description, it sounds like your equipment is all local.
If that’s the case, I’d start with the obvious stuff, like are all of the link lights working. I had the power fail on one of my local PBX servers and it wiped out the ethernet card. In fact, over the years, the number 1 hardware problem I’ve had has been network cards failing.
Log into the console on the PBX. Use the commands
“tcpdump -i etho” and let that run for a couple of minutes. You should see traffic from other hosts.
Next, try
“tcpdump -i eth0 host sbc.local.addr” where the address of the sbc is the sbc.local.addr.
If you get nothing from that machine, but get traffic from other hosts on your network, it’s likely to be the SBC or the switch the SBC is plugged into. If you are getting traffic from the SBC, it’s something software, like a bad routing entry or a bad firewall entry.
Remember, tcpdump sees traffic on the promiscuous port on the machine, so it can see traffic that the application is stopping.
Summary of the pcap this was the only traffic coming in when dialing in from an outside line.
15 2021-07-01 12:48:59.700457 InnoMedi_0a:aa:c8 Broadcast ARP 64 Who has 192.168.1.3? Tell 192.168.1.2
That suggests that 192.168.1.2 is able to transmit. Is an InnoMedia card, with MAC ending in 0a:aa:c8 plausible for the SBC. Maybe it’s ethernet receiver, or receive path, is broken, and it is transmitting blind.
Assuming that the pcap was unfiltered, then the PBX should have replied to those ARP requests, but didn’t. I don’t know enough about the OS internals to know why that might be.
From a root shell prompt, issue arp -i eth0 -s 192.168.1.2 00:10:99:0a:aa:c8
and then try to ping 192.168.1.2
Thanks everyone for all the input. I’d still be scratching my head without it. I got / we got it resolved. It was the switch after all; a Dell PowerEdge 3548P… It used to be set up with a Voice VLAN. This site only has 8 phones. So I deleted the VLAN to simplify things and put everything on the native LAN. Oddly it seem the power outage brought some of the setting back but not all of them. So when I was looking in the switch I didn’t see 2 ports were once again tagged as VLAN100. So I had to reset the switch to factory to get them to clear out. All is well now. I can even ping the SBC now. Thanks again for all the help.