Where to start troubleshooting trunk issues

I am brand new to FreePBX so apologies in advance if this is a silly question.

Background
I recently installed PIAF-Green on a dev VM and signed up for service with SIPStation. The initial configuration was succesful, both incoming and outgoing (bridged) calls worked. However for the last few days I’ve had problems making calls after the box has been idle for a while (don’t know if its a coincidence).

For example last night calls were working fine. Now ten hours later I can’t place calls in either direction. I did an “amportal restart”, but no change.

Dialing into one of my DID’s gives a message “You have reached a non-working number…”. If I try an outbound bridged call (using “Orginate”) the first leg of the call rings, but disconnects with a busy signal.

I checked the logs and see a ton of these messages:

[2013-12-24 08:20:27] NOTICE[23084] chan_sip.c: – Registration for ‘[email protected]’ timed out, trying again (Attempt #1878)

FreePBX status screen shows:

IP Trunks Online: 2
IP Trunk Registrations: 1

A traceroute on trunk1.freepbx.com is successful, but trunk2.freepbx.com times out, which suggests a problem with one of the gateways.

Any ideas on what to try next?

correct

nmap -v trunk2.freepbx.com

says it ain’t there, and a

nmap -v -sU -p 5060 trunk2.freepbx.com

stops at fidelityaccess.com. I would wait until it is fixed as I am sure that route is being monitored and noticed to be not working, if critical you will need to open a ticket with your provider

Thanks for the confirm that it is not just an issue at my end.

This may be a silly question, but shouldn’t calls still work if since one of the gateways is reachable ie trunk1.freepbx.com?

That would depend on how you configured your outbound routes, if they reference both trunks it should gracefully “fail-over” to a working trunk, for inbound you would have to call your provider to see if they do the same thing.

Ah, that makes sense. The two trunks were “auto” set up using SIPStation module:

SIPStation-Out
SIPStation-In

From what I can tell, they seem to be configured to do the fail-over. Anything look amiss here?

SIPStation-Out

Trunk Sequence for Matched Routes
[0] - fpbx-1-xxxxxxxxxxx 
[1] - fpbx-2-xxxxxxxxxxx 


SIPStation-IN

Trunk Sequence for Matched Routes
[0] - fpbx-1-xxxxxxxxxxx 
[1] - fpbx-2-xxxxxxxxxxx 

FWIW: This is a test box. For now, I purchased 4 trunks total from SIPStation. According to their docs they say they auto-configure only the 2 trunks. So as expected, the SIPStation FreePBX module shows 2 trunks configured, and total of 4 channels.

http://wiki.freepbx.org/display/ST/Trunk+Information

Sorry, I don’t use that carrier or that module, but are you sure you need trunks and not just extra channels.

Without doubt trunk2 is not doing what it should (but there is a trunk3). Either way it is a sipstation problem so go there first.

Honestly, I’m not 100% certain. The lingo on their purchase screens does not quite match up with the module screens/documentation. When you purchase it is called a “Trunk”, but in the module screen each one is called a “Channel”:

Currently now the module screen is displaying 4 channels (as I expected). So it seems like it’s configured correctly. But yet … no calls are working. So I’m not sure what to think.

Right now I’m just testing and trying to evaluate the reliability of the SIP service. So before I jump to any conclusions I wanted to rule out possible pilot-errors :wink:

Yeah, I will have a look there. See if I need to open a ticket.

[quote]Sorry, I don’t use that carrier …[/quite]

Even so, your answers have been very helpful, so thanks for that!

(On the up side - they offer a failover phone number for each DID you purchase. I was able to confirm those are working. So at least something positive came out of the service being down.)

call it trunks. sure :wink: Based on what I read in their documentation

I can assure you SIP works very well but only as well as YOUR network and the network of your chosen providers does.

Maybe acquire a second SIP carrier account and have the currently non-functional carrier failover to that other carrier’s DID’s, that covers your bases a little. . . .

Exactly. I’m just trying to get a good enough handle on it to pinpoint whether the problem is external only (sip provider), internal or a combination of both. But a backup provider sounds like a good idea. I’ll have to look around for a recent list of providers.

Cheers

Perhaps start here . . . .

*** Deleted useless confusing link - SkykingOH

( I rescind that, apparently voipinfo has gone sponsored and all/most of those offered are crap . . . , oh well, back to google . . . . )

If you do a tracert between to mirror1.freepbx.org you will see the same thing. You can browse to it however at http://mirror1.freepbx.org/modules

I don’t have application level access to the trunking server but I just got off the phone having Christmas greetings with the President of Schmooze Tony Lewis and he would have mentioned a core application alarm to me.

On the network side, I can tell you that the BGP looking glass sites look fine and we are 100% functional. I am the CTO of the hosting company in Cleveland that hosts the commercial FreePBX stuff and provides space and Bandwidth) for the Open Source and Forum. (Schmooze has other data centers). Since we control the BGP in CLE the firewall is very close to the BGP core router and it doesn’t answer the ICMP messages necessary to complete mappings or trace routes.

The host is alive, our BGP peer with Fidelity is healthy and Looking Glass is perfect from XO in Seattle.

Schmooze_Parma_SSG320-> ping 66.243.108.101 Type escape sequence to abort

Sending 5, 100-byte ICMP Echos to 66.243.108.101, timeout is 1 seconds
!!!
Success Rate is 100 percent (5/5), round-trip time min/avg/max=0/0/1 ms
Schmooze_Parma_SSG320->
Schmooze_Parma_SSG320->
SterlingBGP#sh ip bgp neighbors
BGP neighbor is 74.207.211.73, remote AS 22958, external link
Description: To Fidelity
BGP version 4, remote router ID 66.94.80.9
BGP state = Established, up for 24w1d
Last read 00:00:03, last write 00:00:25, hold time is 90, keepalive interval is 30 seconds
Neighbor capabilities:
Route refresh: advertised and received(old & new)
Address family IPv4 Unicast: advertised and received
Message statistics:
InQ depth is 0
OutQ depth is 0
Sent Rcvd
Opens: 24 24
Notifications: 22 0
Updates: 32 66881966
Keepalives: 705509 744444
Route Refresh: 0 0
Total: 705587 67626434
Default minimum time between advertisement runs is 30 seconds

From XO Looking Glass:

show ip bgp 66.243.108.101
BGP routing table entry for 66.243.108.0/24, version 113662218

      • Advertised to update-groups:
        58
      • Refresh Epoch 1
      • 2828 26554 22958 14001, (received & used)
        65.106.7.253 (metric 12360) from 65.106.7.253 (65.106.7.253)
        Origin IGP, metric 3, localpref 100, valid, external
        Refresh Epoch 1

Level 3 Gives us a thumbs up also in North America with two valid routes in:

Route Server: route-server.gblx.net

BGP routing table entry for 66.243.108.0/24, version 630691714
Paths: (2 available, best #2, table default)

We are having no issues on trunk2. This would appear to be a issue on your side. Yes you register to our 2 trunk servers for redundancy regardless on how many trunks/call paths you buy from us.

Please open a ticket with us on your trunk2 issue but its something on your network/ISP level.

Regardless trunk2 not working right now for you would not stop your service since you have trunk1 registered

I just did some more looking and we had tens of thousands of calls today alone on trunk2 across thousands of customer accounts and I can place and receive calls just fine on it. I also show no drops in registrations in the past 48 hours.

I would appear you have a settings or a network issue on your box somewhere.

No you can not ping trunk2 as its blocked from pings at the edge firewall so Dicko was giving you bad advice and outcome.

I also ask what other carrier has the CTO of the network provider and the President of the carrier respond to a client issue “just to make sure” Surely there is no greater accountability than the Schmooze FreePBX team. Micro Advantage is proud to be a certified integrator of FreePBX and PBXact and network provider to Schmooze.

Merry Christmas!

One other thing to the OP. The tracerouting and networking mapping are not valid tests. Is your server behind a router or firewall that is doing translation (NAT)?

The symptoms you describe sound like a SIP/NAT or some type of SIP ALG/Helper in your firewall.

Sorry for the confusing language. In telco speak many terms are interchangeable. Trunks are pipes between your server and gateway that connects to the cluster of servers that make the service work (functions like switching, routing, billing and provisioning). A trunk can carrier as many calls as your bandwidth can handle and your carrier allows you to create.

You register to the server you want to receive inbound calls at. This tells the network where to send calls to.

Tell us about your network config and we can help you get going. All of us help out on the forums. Please be confident that the Sipstation network has not had any outages.

For those sitting in the bleachers trunk3 is not in production yet and will be for the users west of the Mason Dixon. Official announcements will be made via the blogs.

Thank you both for your detailed responses. Especially on a holiday eve. (A belated Merry Christmas!)

Yes, it behind a firewall. Given that we just set this up in a VM recently, I definitely wasn’t ruling out the possibility of “internal” issues. But I guess I’m not using the right tests. So what would be a reasonable litmus test for determining if a connectivity problem is likely to be external or internal? (So I know where to start troubleshooting issues and am not running around like “Chicken Little”.)

We’re running FreePBX in a VM, behind SonicWall firewall. Today the SIPStation status screen shows all green, but the other day it was showing the Secondary gateway as “Not Registered”. Not sure about the Firewall Test.

SIPStation Status (Today)

Asterisk Registration Status     (Primary) Registered   (Secondary) Registered
SIP Ping                         (Primary) OK           (Secondary) OK
Firewall Test:  [Pass]

Most of our settings are pretty vanilla. However, we noticed calls dropping off at exactly 15 minutes, so we added “session-timers=refuse” to try and resolve the issue.

Asterisk SIP Settings

NAT: Yes
IP Config: Static IP
External IP: xx.xx.xxx.xx  
Local Networks: 
10.48.48.0 / 255.255.255.0
192.168.0.0 / 255.255.255.0

Other SIP Settings:
session-timers = refuse

Our network guy handled the firewall config, as per the docs below, but I can get you more specifics. Just let me know what information you need.

http://wiki.freepbx.org/display/ST/Configuration+of+SIPStation+Trunks

A little more info, I tried dialing into one of my DID’s from a cell phone and land line. Neither connects, but the messages are slightly different:

Land “We’re sorry, your call cannot be completed at this time…”.
Cell: “You have reached a non-working number…”.

Please open a ticket with support.schmoozecom.com and we can look at our logs but SonicWalls are horrible with SIP and the ALG always gets in the way. You need to disable all SIP ALG and SIP Helpers.